[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:22:14.256186 29845 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.29.37.126:34227
I20260709 12:22:14.257652 29845 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:22:14.258671 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:14.274835 29852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.274906 29854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:14.276237 29851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:14.276854 29845 server_base.cc:1061] running on GCE node
I20260709 12:22:15.464924 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:15.465145 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:15.465301 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599735465286 us; error 0 us; skew 500 ppm
I20260709 12:22:15.466069 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:15.471680 29845 webserver.cc:533] Webserver started at http://127.29.37.126:34301/ using document root <none> and password file <none>
I20260709 12:22:15.472596 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:15.472841 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:15.473327 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:15.479255 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/master-0-root/instance:
uuid: "f6ec5e1f8fe34ba59c772c3141235a64"
format_stamp: "Formatted at 2026-07-09 12:22:15 on dist-test-slave-g1m6"
I20260709 12:22:15.487005 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:22:15.491941 29860 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:15.492949 29845 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 12:22:15.493294 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/master-0-root
uuid: "f6ec5e1f8fe34ba59c772c3141235a64"
format_stamp: "Formatted at 2026-07-09 12:22:15 on dist-test-slave-g1m6"
I20260709 12:22:15.493615 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:15.549113 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:15.550669 29845 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:22:15.551119 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:15.621192 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.126:34227
I20260709 12:22:15.621256 29911 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.126:34227 every 8 connection(s)
I20260709 12:22:15.629066 29912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:15.650049 29912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64: Bootstrap starting.
I20260709 12:22:15.655893 29912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:15.657706 29912 log.cc:826] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64: Log is configured to *not* fsync() on all Append() calls
I20260709 12:22:15.662063 29912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64: No bootstrap required, opened a new log
I20260709 12:22:15.682367 29912 raft_consensus.cc:359] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec5e1f8fe34ba59c772c3141235a64" member_type: VOTER }
I20260709 12:22:15.682880 29912 raft_consensus.cc:385] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:15.683144 29912 raft_consensus.cc:740] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ec5e1f8fe34ba59c772c3141235a64, State: Initialized, Role: FOLLOWER
I20260709 12:22:15.684137 29912 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec5e1f8fe34ba59c772c3141235a64" member_type: VOTER }
I20260709 12:22:15.684626 29912 raft_consensus.cc:399] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:15.684933 29912 raft_consensus.cc:493] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:15.685276 29912 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:15.689313 29912 raft_consensus.cc:515] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec5e1f8fe34ba59c772c3141235a64" member_type: VOTER }
I20260709 12:22:15.690034 29912 leader_election.cc:304] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6ec5e1f8fe34ba59c772c3141235a64; no voters: 
I20260709 12:22:15.691772 29912 leader_election.cc:290] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:15.692185 29915 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:15.694640 29915 raft_consensus.cc:697] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 1 LEADER]: Becoming Leader. State: Replica: f6ec5e1f8fe34ba59c772c3141235a64, State: Running, Role: LEADER
I20260709 12:22:15.695492 29915 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec5e1f8fe34ba59c772c3141235a64" member_type: VOTER }
I20260709 12:22:15.696341 29912 sys_catalog.cc:565] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:15.706269 29916 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6ec5e1f8fe34ba59c772c3141235a64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec5e1f8fe34ba59c772c3141235a64" member_type: VOTER } }
I20260709 12:22:15.706945 29916 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:15.707440 29917 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6ec5e1f8fe34ba59c772c3141235a64. Latest consensus state: current_term: 1 leader_uuid: "f6ec5e1f8fe34ba59c772c3141235a64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ec5e1f8fe34ba59c772c3141235a64" member_type: VOTER } }
I20260709 12:22:15.708114 29917 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:15.716197 29925 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:15.727439 29925 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:15.729673 29845 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:15.743785 29925 catalog_manager.cc:1383] Generated new cluster ID: 7cbd6d4655114b8eaaf72d53579666ce
I20260709 12:22:15.744041 29925 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:15.759851 29925 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:15.761302 29925 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:15.775720 29925 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64: Generated new TSK 0
I20260709 12:22:15.776546 29925 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:15.797088 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:15.806478 29934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:15.808883 29935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:15.812594 29937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:15.812884 29845 server_base.cc:1061] running on GCE node
I20260709 12:22:15.813635 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:15.813825 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:15.814041 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599735814008 us; error 0 us; skew 500 ppm
I20260709 12:22:15.814670 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:15.818511 29845 webserver.cc:533] Webserver started at http://127.29.37.65:45375/ using document root <none> and password file <none>
I20260709 12:22:15.819043 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:15.819239 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:15.819530 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:15.820598 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-0-root/instance:
uuid: "bd46e724d7564358bf7e8d707dccab8e"
format_stamp: "Formatted at 2026-07-09 12:22:15 on dist-test-slave-g1m6"
I20260709 12:22:15.826316 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 12:22:15.830294 29942 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:15.831081 29845 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:22:15.831377 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "bd46e724d7564358bf7e8d707dccab8e"
format_stamp: "Formatted at 2026-07-09 12:22:15 on dist-test-slave-g1m6"
I20260709 12:22:15.831676 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:15.850411 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:15.851720 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:15.854434 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:15.858407 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:15.858611 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:15.858922 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:15.859073 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:15.962972 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:38627
I20260709 12:22:15.963057 30004 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:38627 every 8 connection(s)
I20260709 12:22:15.969801 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:15.979684 30009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:15.984499 30010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:15.993374 29845 server_base.cc:1061] running on GCE node
W20260709 12:22:15.994310 30013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:15.995278 30005 heartbeater.cc:344] Connected to a master server at 127.29.37.126:34227
I20260709 12:22:15.995402 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:22:15.995729 30005 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:15.996695 30005 heartbeater.cc:507] Master 127.29.37.126:34227 requested a full tablet report, sending...
W20260709 12:22:15.995697 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:15.997793 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599735997770 us; error 0 us; skew 500 ppm
I20260709 12:22:15.998724 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:15.999884 29877 ts_manager.cc:194] Registered new tserver with Master: bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627)
I20260709 12:22:16.002681 29877 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60422
I20260709 12:22:16.004930 29845 webserver.cc:533] Webserver started at http://127.29.37.66:39483/ using document root <none> and password file <none>
I20260709 12:22:16.005472 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:16.005698 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:16.005991 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:16.007077 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-1-root/instance:
uuid: "5875c6c1c8024f41b05b9c8965c184f8"
format_stamp: "Formatted at 2026-07-09 12:22:16 on dist-test-slave-g1m6"
I20260709 12:22:16.012253 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.001s
I20260709 12:22:16.016124 30017 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:16.016906 29845 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:22:16.017184 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-1-root
uuid: "5875c6c1c8024f41b05b9c8965c184f8"
format_stamp: "Formatted at 2026-07-09 12:22:16 on dist-test-slave-g1m6"
I20260709 12:22:16.017463 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:16.028108 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:16.029423 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:16.031535 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:16.034526 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:16.034752 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:16.034986 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:16.035151 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:16.131510 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.66:37929
I20260709 12:22:16.131629 30079 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.66:37929 every 8 connection(s)
I20260709 12:22:16.140699 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:16.149008 30083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:16.153088 30084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:16.153860 30080 heartbeater.cc:344] Connected to a master server at 127.29.37.126:34227
I20260709 12:22:16.154326 30080 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:16.155427 30080 heartbeater.cc:507] Master 127.29.37.126:34227 requested a full tablet report, sending...
I20260709 12:22:16.157964 29877 ts_manager.cc:194] Registered new tserver with Master: 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
W20260709 12:22:16.158607 30086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:16.159368 29877 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60424
I20260709 12:22:16.161610 29845 server_base.cc:1061] running on GCE node
I20260709 12:22:16.162513 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:16.162715 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:16.162889 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599736162855 us; error 0 us; skew 500 ppm
I20260709 12:22:16.163424 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:16.165781 29845 webserver.cc:533] Webserver started at http://127.29.37.67:42313/ using document root <none> and password file <none>
I20260709 12:22:16.166306 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:16.166499 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:16.166772 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:16.167850 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-2-root/instance:
uuid: "f7280ec9528e47eabc07fe33f45895d7"
format_stamp: "Formatted at 2026-07-09 12:22:16 on dist-test-slave-g1m6"
I20260709 12:22:16.172289 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:16.175881 30091 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:16.176690 29845 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:22:16.176968 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-2-root
uuid: "f7280ec9528e47eabc07fe33f45895d7"
format_stamp: "Formatted at 2026-07-09 12:22:16 on dist-test-slave-g1m6"
I20260709 12:22:16.177225 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599734091681-29845-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:16.206071 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:16.207567 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:16.208976 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:16.211572 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:16.211750 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:16.212033 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:16.212266 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:16.312767 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.67:43943
I20260709 12:22:16.312870 30153 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.67:43943 every 8 connection(s)
I20260709 12:22:16.329587 30154 heartbeater.cc:344] Connected to a master server at 127.29.37.126:34227
I20260709 12:22:16.330003 30154 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:16.330865 30154 heartbeater.cc:507] Master 127.29.37.126:34227 requested a full tablet report, sending...
I20260709 12:22:16.332897 29877 ts_manager.cc:194] Registered new tserver with Master: f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:16.333468 29845 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015714968s
I20260709 12:22:16.334374 29877 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60432
I20260709 12:22:16.382648 29877 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60444:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:16.386072 29877 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:16.474772 30045 tablet_service.cc:1511] Processing CreateTablet for tablet 1cda325da7a94860a205195db11e9949 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:16.476440 30045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cda325da7a94860a205195db11e9949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.498993 30044 tablet_service.cc:1511] Processing CreateTablet for tablet 756a83b7a17a461d9ff9771f3f411793 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:16.500339 30044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756a83b7a17a461d9ff9771f3f411793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.502856 30043 tablet_service.cc:1511] Processing CreateTablet for tablet 03abc00bdfec46bd817f6b535c1b4650 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:16.504134 30043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03abc00bdfec46bd817f6b535c1b4650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.504796 30042 tablet_service.cc:1511] Processing CreateTablet for tablet 96f0adea973d4d23918b329c2b88c852 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:16.506104 30042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96f0adea973d4d23918b329c2b88c852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.515915 30041 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd34ffceb744b67a451d4a03c5d5c82 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:16.517242 30041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd34ffceb744b67a451d4a03c5d5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.532613 30119 tablet_service.cc:1511] Processing CreateTablet for tablet 1cda325da7a94860a205195db11e9949 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:16.528184 30118 tablet_service.cc:1511] Processing CreateTablet for tablet 756a83b7a17a461d9ff9771f3f411793 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:16.528242 30040 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6f8015ea804df0bd92ac6cfb257d29 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:16.541931 30044 tablet_service.cc:1511] Processing CreateTablet for tablet eb6e17441f6747c0a655166876508d7f (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:16.543042 30116 tablet_service.cc:1511] Processing CreateTablet for tablet 96f0adea973d4d23918b329c2b88c852 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:16.543859 30112 tablet_service.cc:1511] Processing CreateTablet for tablet eb6e17441f6747c0a655166876508d7f (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:16.550063 30044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6e17441f6747c0a655166876508d7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.548714 30115 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd34ffceb744b67a451d4a03c5d5c82 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:16.552937 29970 tablet_service.cc:1511] Processing CreateTablet for tablet 1cda325da7a94860a205195db11e9949 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:16.550094 30117 tablet_service.cc:1511] Processing CreateTablet for tablet 03abc00bdfec46bd817f6b535c1b4650 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:16.548882 30039 tablet_service.cc:1511] Processing CreateTablet for tablet f6e1c832aab64224b83c0aaa7dab64fd (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:16.559983 30040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6f8015ea804df0bd92ac6cfb257d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.558173 30119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cda325da7a94860a205195db11e9949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.558820 30043 tablet_service.cc:1511] Processing CreateTablet for tablet 8c934a48601e4629823e2c6dc17eaad6 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:16.559695 30171 tablet_bootstrap.cc:492] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:16.562191 29970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cda325da7a94860a205195db11e9949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.562978 30111 tablet_service.cc:1511] Processing CreateTablet for tablet 8c934a48601e4629823e2c6dc17eaad6 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:16.563149 30112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6e17441f6747c0a655166876508d7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.554183 30113 tablet_service.cc:1511] Processing CreateTablet for tablet f6e1c832aab64224b83c0aaa7dab64fd (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:16.568692 30114 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6f8015ea804df0bd92ac6cfb257d29 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:16.574285 29969 tablet_service.cc:1511] Processing CreateTablet for tablet 756a83b7a17a461d9ff9771f3f411793 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:16.571157 29967 tablet_service.cc:1511] Processing CreateTablet for tablet 96f0adea973d4d23918b329c2b88c852 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:16.578066 29969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756a83b7a17a461d9ff9771f3f411793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.583566 29967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96f0adea973d4d23918b329c2b88c852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.582180 30118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756a83b7a17a461d9ff9771f3f411793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.584584 29968 tablet_service.cc:1511] Processing CreateTablet for tablet 03abc00bdfec46bd817f6b535c1b4650 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:16.579639 29966 tablet_service.cc:1511] Processing CreateTablet for tablet 1cd34ffceb744b67a451d4a03c5d5c82 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:16.591851 29964 tablet_service.cc:1511] Processing CreateTablet for tablet f6e1c832aab64224b83c0aaa7dab64fd (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:16.590183 29963 tablet_service.cc:1511] Processing CreateTablet for tablet eb6e17441f6747c0a655166876508d7f (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:16.590312 30114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6f8015ea804df0bd92ac6cfb257d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.595170 30043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c934a48601e4629823e2c6dc17eaad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.584573 29965 tablet_service.cc:1511] Processing CreateTablet for tablet 6d6f8015ea804df0bd92ac6cfb257d29 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:16.599296 30039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e1c832aab64224b83c0aaa7dab64fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.602104 29968 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03abc00bdfec46bd817f6b535c1b4650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.602566 30111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c934a48601e4629823e2c6dc17eaad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.604275 29964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e1c832aab64224b83c0aaa7dab64fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.602499 29962 tablet_service.cc:1511] Processing CreateTablet for tablet 8c934a48601e4629823e2c6dc17eaad6 (DEFAULT_TABLE table=test-workload [id=71e505ddfd6248309239d70c6a42f5d0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:16.607251 29966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd34ffceb744b67a451d4a03c5d5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.609961 29965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d6f8015ea804df0bd92ac6cfb257d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.607844 30117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03abc00bdfec46bd817f6b535c1b4650. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.614084 29962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c934a48601e4629823e2c6dc17eaad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.618849 29963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb6e17441f6747c0a655166876508d7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.650300 30116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96f0adea973d4d23918b329c2b88c852. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.663540 30115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cd34ffceb744b67a451d4a03c5d5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:16.667501 30113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6e1c832aab64224b83c0aaa7dab64fd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:16.720396 29924 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:16.729681 30171 tablet_bootstrap.cc:654] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.774969 30172 tablet_bootstrap.cc:492] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
I20260709 12:22:16.797497 30171 tablet_bootstrap.cc:492] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:16.798000 30171 ts_tablet_manager.cc:1403] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.264s	user 0.015s	sys 0.016s
I20260709 12:22:16.805730 30172 tablet_bootstrap.cc:654] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.814465 30173 tablet_bootstrap.cc:492] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:16.824088 30172 tablet_bootstrap.cc:492] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:16.824566 30172 ts_tablet_manager.cc:1403] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.000s
I20260709 12:22:16.817641 30171 raft_consensus.cc:359] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.826807 30171 raft_consensus.cc:385] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.827183 30171 raft_consensus.cc:740] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.827351 30173 tablet_bootstrap.cc:654] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.827420 30172 raft_consensus.cc:359] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.828312 30172 raft_consensus.cc:385] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.828342 30171 consensus_queue.cc:260] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.829066 30172 raft_consensus.cc:740] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.829909 30172 consensus_queue.cc:260] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.839160 30005 heartbeater.cc:499] Master 127.29.37.126:34227 was elected leader, sending a full tablet report...
I20260709 12:22:16.841348 30173 tablet_bootstrap.cc:492] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:16.841889 30173 ts_tablet_manager.cc:1403] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.007s
I20260709 12:22:16.843983 30172 ts_tablet_manager.cc:1434] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.005s
I20260709 12:22:16.844977 30172 tablet_bootstrap.cc:492] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
I20260709 12:22:16.849584 30080 heartbeater.cc:499] Master 127.29.37.126:34227 was elected leader, sending a full tablet report...
I20260709 12:22:16.849593 30179 raft_consensus.cc:493] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.850170 30179 raft_consensus.cc:515] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.851163 30172 tablet_bootstrap.cc:654] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.853060 30179 leader_election.cc:290] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:16.857630 30171 ts_tablet_manager.cc:1434] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.006s
I20260709 12:22:16.858560 30171 tablet_bootstrap.cc:492] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:16.863490 30173 raft_consensus.cc:359] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.868588 30173 raft_consensus.cc:385] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.869339 30173 raft_consensus.cc:740] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.870571 30173 consensus_queue.cc:260] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.865864 30171 tablet_bootstrap.cc:654] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.878669 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:16.881528 30172 tablet_bootstrap.cc:492] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:16.881981 30172 ts_tablet_manager.cc:1403] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.037s	user 0.003s	sys 0.026s
I20260709 12:22:16.887256 30172 raft_consensus.cc:359] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.888612 30172 raft_consensus.cc:385] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.889048 30172 raft_consensus.cc:740] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.889954 30172 consensus_queue.cc:260] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.892786 30172 ts_tablet_manager.cc:1434] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:22:16.893657 30172 tablet_bootstrap.cc:492] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
W20260709 12:22:16.897794 30081 tablet.cc:2406] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260709 12:22:16.898504 30019 leader_election.cc:343] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:16.902278 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:16.903319 30173 ts_tablet_manager.cc:1434] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.007s
W20260709 12:22:16.903967 30019 leader_election.cc:343] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:16.904277 30173 tablet_bootstrap.cc:492] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:16.904402 30019 leader_election.cc:304] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8; no voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:16.905392 30176 raft_consensus.cc:2749] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:16.905562 30154 heartbeater.cc:499] Master 127.29.37.126:34227 was elected leader, sending a full tablet report...
I20260709 12:22:16.913017 30172 tablet_bootstrap.cc:654] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.917312 30173 tablet_bootstrap.cc:654] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.923127 30172 tablet_bootstrap.cc:492] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:16.923658 30172 ts_tablet_manager.cc:1403] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.000s
I20260709 12:22:16.926723 30172 raft_consensus.cc:359] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.930560 30172 raft_consensus.cc:385] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.931270 30172 raft_consensus.cc:740] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.932371 30172 consensus_queue.cc:260] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.934336 30173 tablet_bootstrap.cc:492] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:16.934763 30172 ts_tablet_manager.cc:1434] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:22:16.934790 30173 ts_tablet_manager.cc:1403] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.011s
I20260709 12:22:16.935933 30172 tablet_bootstrap.cc:492] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
I20260709 12:22:16.937884 30173 raft_consensus.cc:359] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.931480 30171 tablet_bootstrap.cc:492] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:16.938621 30173 raft_consensus.cc:385] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.938903 30171 ts_tablet_manager.cc:1403] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.081s	user 0.011s	sys 0.016s
I20260709 12:22:16.939045 30173 raft_consensus.cc:740] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.939898 30173 consensus_queue.cc:260] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.942170 30173 ts_tablet_manager.cc:1434] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:16.943096 30173 tablet_bootstrap.cc:492] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:16.943465 30172 tablet_bootstrap.cc:654] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.949250 30173 tablet_bootstrap.cc:654] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.957316 30184 raft_consensus.cc:493] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.957820 30184 raft_consensus.cc:515] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.960424 30184 leader_election.cc:290] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:16.964346 30173 tablet_bootstrap.cc:492] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:16.964828 30173 ts_tablet_manager.cc:1403] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.005s
I20260709 12:22:16.965608 30172 tablet_bootstrap.cc:492] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:16.968137 30173 raft_consensus.cc:359] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.968809 30172 ts_tablet_manager.cc:1403] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.011s
I20260709 12:22:16.969000 30173 raft_consensus.cc:385] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.969322 30173 raft_consensus.cc:740] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.970084 30173 consensus_queue.cc:260] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.972488 30173 ts_tablet_manager.cc:1434] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:22:16.973387 30173 tablet_bootstrap.cc:492] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:16.974478 30171 raft_consensus.cc:359] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.972215 30172 raft_consensus.cc:359] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.977470 30172 raft_consensus.cc:385] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.977852 30172 raft_consensus.cc:740] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.978611 30172 consensus_queue.cc:260] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.980244 30173 tablet_bootstrap.cc:654] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.975224 30171 raft_consensus.cc:385] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.985245 30172 ts_tablet_manager.cc:1434] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:22:16.989894 30172 tablet_bootstrap.cc:492] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
I20260709 12:22:16.989748 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:16.993438 30173 tablet_bootstrap.cc:492] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:16.993878 30173 ts_tablet_manager.cc:1403] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.000s
I20260709 12:22:16.996714 30173 raft_consensus.cc:359] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:16.997400 30172 tablet_bootstrap.cc:654] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:16.997382 30173 raft_consensus.cc:385] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:16.997907 30173 raft_consensus.cc:740] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
W20260709 12:22:16.998375 30093 leader_election.cc:343] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:16.985656 30171 raft_consensus.cc:740] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:16.998684 30173 consensus_queue.cc:260] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.001006 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
W20260709 12:22:17.002383 30093 leader_election.cc:343] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.002269 30171 consensus_queue.cc:260] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.002893 30173 ts_tablet_manager.cc:1434] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:22:17.002811 30093 leader_election.cc:304] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7280ec9528e47eabc07fe33f45895d7; no voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e
I20260709 12:22:17.003587 30184 raft_consensus.cc:2749] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:17.004702 30171 ts_tablet_manager.cc:1434] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.065s	user 0.004s	sys 0.003s
I20260709 12:22:17.005321 30173 tablet_bootstrap.cc:492] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:17.005769 30171 tablet_bootstrap.cc:492] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:17.013458 30173 tablet_bootstrap.cc:654] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.018139 30171 tablet_bootstrap.cc:654] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.022843 30172 tablet_bootstrap.cc:492] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:17.023423 30172 ts_tablet_manager.cc:1403] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.001s
I20260709 12:22:17.023986 30173 tablet_bootstrap.cc:492] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:17.024415 30173 ts_tablet_manager.cc:1403] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 12:22:17.026984 30172 raft_consensus.cc:359] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.027740 30172 raft_consensus.cc:385] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.027966 30171 tablet_bootstrap.cc:492] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:17.027551 30173 raft_consensus.cc:359] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.028115 30172 raft_consensus.cc:740] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.028501 30173 raft_consensus.cc:385] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.028743 30171 ts_tablet_manager.cc:1403] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.003s
I20260709 12:22:17.028831 30173 raft_consensus.cc:740] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.028960 30172 consensus_queue.cc:260] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.029846 30173 consensus_queue.cc:260] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.031735 30172 ts_tablet_manager.cc:1434] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:17.032137 30173 ts_tablet_manager.cc:1434] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:22:17.032629 30172 tablet_bootstrap.cc:492] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
I20260709 12:22:17.033007 30173 tablet_bootstrap.cc:492] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:17.039249 30173 tablet_bootstrap.cc:654] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.039786 30172 tablet_bootstrap.cc:654] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.054692 30173 tablet_bootstrap.cc:492] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:17.055130 30173 ts_tablet_manager.cc:1403] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.016s
I20260709 12:22:17.057760 30173 raft_consensus.cc:359] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.058455 30173 raft_consensus.cc:385] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.058738 30173 raft_consensus.cc:740] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.059427 30173 consensus_queue.cc:260] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.061818 30173 ts_tablet_manager.cc:1434] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:17.062692 30173 tablet_bootstrap.cc:492] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:17.067688 30171 raft_consensus.cc:359] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.068465 30171 raft_consensus.cc:385] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.068815 30171 raft_consensus.cc:740] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.069169 30173 tablet_bootstrap.cc:654] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.072592 30172 tablet_bootstrap.cc:492] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:17.073056 30172 ts_tablet_manager.cc:1403] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.041s	user 0.025s	sys 0.008s
I20260709 12:22:17.069681 30171 consensus_queue.cc:260] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.075059 30171 ts_tablet_manager.cc:1434] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:22:17.076721 30171 tablet_bootstrap.cc:492] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:17.079157 30172 raft_consensus.cc:359] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.079942 30172 raft_consensus.cc:385] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.080235 30172 raft_consensus.cc:740] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.080557 30173 tablet_bootstrap.cc:492] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:17.080986 30173 ts_tablet_manager.cc:1403] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 12:22:17.080976 30172 consensus_queue.cc:260] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.083590 30172 ts_tablet_manager.cc:1434] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:22:17.083990 30173 raft_consensus.cc:359] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.084511 30172 tablet_bootstrap.cc:492] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
I20260709 12:22:17.084628 30173 raft_consensus.cc:385] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.084972 30173 raft_consensus.cc:740] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.085624 30173 consensus_queue.cc:260] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.087945 30173 ts_tablet_manager.cc:1434] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:17.088809 30173 tablet_bootstrap.cc:492] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:17.095018 30173 tablet_bootstrap.cc:654] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.103420 30172 tablet_bootstrap.cc:654] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.107257 30171 tablet_bootstrap.cc:654] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.113898 30173 tablet_bootstrap.cc:492] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:17.114327 30173 ts_tablet_manager.cc:1403] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.007s
I20260709 12:22:17.116595 30173 raft_consensus.cc:359] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.117201 30173 raft_consensus.cc:385] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.117444 30173 raft_consensus.cc:740] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.118052 30173 consensus_queue.cc:260] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.120184 30173 ts_tablet_manager.cc:1434] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:22:17.121527 30173 tablet_bootstrap.cc:492] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7: Bootstrap starting.
I20260709 12:22:17.122848 30184 raft_consensus.cc:493] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.123276 30184 raft_consensus.cc:515] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.124210 30172 tablet_bootstrap.cc:492] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:17.125165 30184 leader_election.cc:290] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:17.124671 30172 ts_tablet_manager.cc:1403] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.003s
I20260709 12:22:17.126250 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.127804 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:17.128814 30172 raft_consensus.cc:359] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.129483 30172 raft_consensus.cc:385] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.129737 30173 tablet_bootstrap.cc:654] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.129784 30172 raft_consensus.cc:740] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
W20260709 12:22:17.129782 30093 leader_election.cc:343] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:17.131419 30093 leader_election.cc:343] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.131196 30172 consensus_queue.cc:260] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.131803 30093 leader_election.cc:304] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7280ec9528e47eabc07fe33f45895d7; no voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e
I20260709 12:22:17.132503 30184 raft_consensus.cc:2749] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:17.135643 30172 ts_tablet_manager.cc:1434] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:17.136588 30172 tablet_bootstrap.cc:492] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
I20260709 12:22:17.141794 30173 tablet_bootstrap.cc:492] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7: No bootstrap required, opened a new log
I20260709 12:22:17.142198 30173 ts_tablet_manager.cc:1403] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.006s
I20260709 12:22:17.143369 30172 tablet_bootstrap.cc:654] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.144569 30173 raft_consensus.cc:359] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.145190 30173 raft_consensus.cc:385] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.145432 30173 raft_consensus.cc:740] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.146044 30173 consensus_queue.cc:260] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.148228 30173 ts_tablet_manager.cc:1434] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 12:22:17.153806 30171 tablet_bootstrap.cc:492] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:17.154268 30171 ts_tablet_manager.cc:1403] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.078s	user 0.006s	sys 0.011s
I20260709 12:22:17.156988 30171 raft_consensus.cc:359] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.157649 30171 raft_consensus.cc:385] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.157943 30171 raft_consensus.cc:740] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.158814 30184 raft_consensus.cc:493] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.159248 30184 raft_consensus.cc:515] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.161257 30184 leader_election.cc:290] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:17.161801 30172 tablet_bootstrap.cc:492] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:17.158640 30171 consensus_queue.cc:260] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.162240 30172 ts_tablet_manager.cc:1403] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.008s
I20260709 12:22:17.162678 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:17.163712 30171 ts_tablet_manager.cc:1434] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:22:17.163765 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.164496 30055 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 0.
I20260709 12:22:17.164708 30171 tablet_bootstrap.cc:492] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:17.164726 30172 raft_consensus.cc:359] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.165593 30172 raft_consensus.cc:385] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.165875 30172 raft_consensus.cc:740] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.166249 30184 raft_consensus.cc:493] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.166676 30184 raft_consensus.cc:515] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.166751 30172 consensus_queue.cc:260] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.168614 30184 leader_election.cc:290] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:17.169029 30172 ts_tablet_manager.cc:1434] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:22:17.169886 30172 tablet_bootstrap.cc:492] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e: Bootstrap starting.
I20260709 12:22:17.173704 30171 tablet_bootstrap.cc:654] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.175485 30093 leader_election.cc:304] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.176230 30184 raft_consensus.cc:2804] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:22:17.176367 30093 leader_election.cc:343] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.178756 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c934a48601e4629823e2c6dc17eaad6" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:17.190954 29980 raft_consensus.cc:2468] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 0.
I20260709 12:22:17.179958 30172 tablet_bootstrap.cc:654] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.176563 30184 raft_consensus.cc:493] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.192167 30184 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.186667 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c934a48601e4629823e2c6dc17eaad6" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.189610 30178 raft_consensus.cc:493] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.192948 30178 raft_consensus.cc:515] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.192195 30093 leader_election.cc:304] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.197002 30178 leader_election.cc:290] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
W20260709 12:22:17.204790 30093 leader_election.cc:343] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.205596 30204 raft_consensus.cc:2804] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:17.206166 30204 raft_consensus.cc:493] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.205832 30184 raft_consensus.cc:515] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.208122 30172 tablet_bootstrap.cc:492] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e: No bootstrap required, opened a new log
I20260709 12:22:17.208552 30172 ts_tablet_manager.cc:1403] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.009s
I20260709 12:22:17.209216 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:17.209561 30171 tablet_bootstrap.cc:492] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:17.210179 30171 ts_tablet_manager.cc:1403] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.017s
I20260709 12:22:17.210705 30184 leader_election.cc:290] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:17.211475 30172 raft_consensus.cc:359] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.206658 30204 raft_consensus.cc:3060] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.214444 30172 raft_consensus.cc:385] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.214747 30172 raft_consensus.cc:740] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.212086 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
W20260709 12:22:17.212926 30093 leader_election.cc:343] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.215685 30055 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.215477 30172 consensus_queue.cc:260] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.213598 30171 raft_consensus.cc:359] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.216249 30171 raft_consensus.cc:385] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.216542 30171 raft_consensus.cc:740] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.217525 30172 ts_tablet_manager.cc:1434] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:17.217264 30171 consensus_queue.cc:260] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.221370 30171 ts_tablet_manager.cc:1434] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:22:17.222288 30171 tablet_bootstrap.cc:492] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:17.227276 30204 raft_consensus.cc:515] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.224205 30055 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 1.
I20260709 12:22:17.230041 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c934a48601e4629823e2c6dc17eaad6" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:17.230381 30093 leader_election.cc:304] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: bd46e724d7564358bf7e8d707dccab8e
I20260709 12:22:17.230652 29980 raft_consensus.cc:3060] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.232080 30184 raft_consensus.cc:2804] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.237648 29980 raft_consensus.cc:2468] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 1.
I20260709 12:22:17.239275 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c934a48601e4629823e2c6dc17eaad6" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
W20260709 12:22:17.240569 30093 leader_election.cc:343] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.244782 30171 tablet_bootstrap.cc:654] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.244894 30093 leader_election.cc:304] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:17.238101 30204 leader_election.cc:290] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:17.247344 30204 raft_consensus.cc:2804] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.247797 30204 raft_consensus.cc:697] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Becoming Leader. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:17.248602 30204 consensus_queue.cc:237] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.261685 30171 tablet_bootstrap.cc:492] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:17.262243 30171 ts_tablet_manager.cc:1403] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.001s
I20260709 12:22:17.262182 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.265300 30171 raft_consensus.cc:359] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.266036 30171 raft_consensus.cc:385] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.266474 30171 raft_consensus.cc:740] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.267194 30171 consensus_queue.cc:260] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.270918 30171 ts_tablet_manager.cc:1434] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:22:17.271840 30171 tablet_bootstrap.cc:492] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:17.278203 30184 raft_consensus.cc:697] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Becoming Leader. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
W20260709 12:22:17.278667 29944 leader_election.cc:343] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.279212 30184 consensus_queue.cc:237] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.280506 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.281217 30129 raft_consensus.cc:2468] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.281674 30178 raft_consensus.cc:493] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.278187 30171 tablet_bootstrap.cc:654] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.282106 30178 raft_consensus.cc:515] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.283381 29944 leader_election.cc:304] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:17.286094 30178 leader_election.cc:290] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.286715 30178 raft_consensus.cc:493] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.287225 30178 raft_consensus.cc:515] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.289506 30178 leader_election.cc:290] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.290113 30178 raft_consensus.cc:2804] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:17.290529 30178 raft_consensus.cc:493] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.290944 30178 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.301873 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e17441f6747c0a655166876508d7f" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.302513 30055 raft_consensus.cc:2468] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.304219 30204 raft_consensus.cc:493] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.304481 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e17441f6747c0a655166876508d7f" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.305079 30129 raft_consensus.cc:2468] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.304801 30204 raft_consensus.cc:515] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.311252 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.311863 30055 raft_consensus.cc:2468] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.312923 30204 leader_election.cc:290] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627)
I20260709 12:22:17.311372 30178 raft_consensus.cc:515] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.313849 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.314437 30055 raft_consensus.cc:2468] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 0.
I20260709 12:22:17.315886 30178 leader_election.cc:290] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.315917 30093 leader_election.cc:304] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.316316 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:17.319626 29980 raft_consensus.cc:2468] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 0.
I20260709 12:22:17.320716 30184 raft_consensus.cc:2804] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:17.321061 30184 raft_consensus.cc:493] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.321393 30184 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.321552 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.324659 29877 catalog_manager.cc:5697] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "f7280ec9528e47eabc07fe33f45895d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:17.326447 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.326725 30184 raft_consensus.cc:515] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.327972 30129 raft_consensus.cc:2393] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd46e724d7564358bf7e8d707dccab8e in current term 1: Already voted for candidate f7280ec9528e47eabc07fe33f45895d7 in this term.
I20260709 12:22:17.328501 30171 tablet_bootstrap.cc:492] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:17.329190 30171 ts_tablet_manager.cc:1403] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.013s
I20260709 12:22:17.329955 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.331581 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:17.331758 30055 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.332142 29980 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.334115 30178 raft_consensus.cc:493] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.334638 30178 raft_consensus.cc:515] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.336768 30178 leader_election.cc:290] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.333822 30171 raft_consensus.cc:359] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.342417 30171 raft_consensus.cc:385] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.342742 30171 raft_consensus.cc:740] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.338822 29980 raft_consensus.cc:2468] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 1.
I20260709 12:22:17.338770 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.343600 30129 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.345356 30055 raft_consensus.cc:2468] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 1.
I20260709 12:22:17.346621 30093 leader_election.cc:304] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.347688 30204 raft_consensus.cc:2804] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.348152 30178 raft_consensus.cc:493] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.348301 30204 raft_consensus.cc:697] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Becoming Leader. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:17.348608 30178 raft_consensus.cc:515] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.349440 30204 consensus_queue.cc:237] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.350589 30178 leader_election.cc:290] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.339300 30184 leader_election.cc:290] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 1 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627)
I20260709 12:22:17.350581 30129 raft_consensus.cc:2468] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.352031 30171 consensus_queue.cc:260] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.339818 29944 leader_election.cc:304] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.353497 29944 leader_election.cc:304] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.354252 30178 raft_consensus.cc:2804] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:17.354588 30178 raft_consensus.cc:493] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.354912 30178 raft_consensus.cc:3060] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.355114 29944 leader_election.cc:304] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:17.355790 30213 raft_consensus.cc:2804] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.354898 30171 ts_tablet_manager.cc:1434] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
W20260709 12:22:17.356869 29944 leader_election.cc:343] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.356832 30212 raft_consensus.cc:2764] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:22:17.358112 30171 tablet_bootstrap.cc:492] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:17.360368 30054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.361023 30054 raft_consensus.cc:2468] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.361847 30212 raft_consensus.cc:493] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.363538 30212 raft_consensus.cc:515] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.366568 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.367372 30213 raft_consensus.cc:697] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.367678 30212 leader_election.cc:290] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.368772 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.369269 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.370296 30129 raft_consensus.cc:2468] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.370829 30128 raft_consensus.cc:2468] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.372030 30178 raft_consensus.cc:515] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.371821 30213 consensus_queue.cc:237] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.368963 29944 leader_election.cc:304] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: 
I20260709 12:22:17.382638 30212 raft_consensus.cc:493] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.383792 30212 raft_consensus.cc:515] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.385655 30212 leader_election.cc:290] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.387570 30218 raft_consensus.cc:2804] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:17.387959 30218 raft_consensus.cc:493] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.386940 30178 leader_election.cc:290] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.388268 30218 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.388595 30171 tablet_bootstrap.cc:654] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.391108 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.392760 30055 raft_consensus.cc:2468] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.393447 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.393895 29944 leader_election.cc:304] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.394044 30129 raft_consensus.cc:2468] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.394634 30054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.395303 30054 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.396083 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e17441f6747c0a655166876508d7f" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.397557 30055 raft_consensus.cc:3060] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.396086 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e17441f6747c0a655166876508d7f" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.398283 30128 raft_consensus.cc:3060] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.396385 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.400743 29944 leader_election.cc:304] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.401417 30176 raft_consensus.cc:493] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.401906 30176 raft_consensus.cc:515] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.402885 30212 raft_consensus.cc:2804] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:17.403537 30129 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 0.
I20260709 12:22:17.403811 30212 raft_consensus.cc:493] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.404299 30212 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.405201 30178 raft_consensus.cc:2804] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:17.405608 30178 raft_consensus.cc:493] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.401939 30218 raft_consensus.cc:515] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.405985 30178 raft_consensus.cc:3060] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.410233 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c934a48601e4629823e2c6dc17eaad6" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:17.411343 29980 raft_consensus.cc:2393] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5875c6c1c8024f41b05b9c8965c184f8 in current term 1: Already voted for candidate f7280ec9528e47eabc07fe33f45895d7 in this term.
I20260709 12:22:17.418044 30176 leader_election.cc:290] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.419860 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c934a48601e4629823e2c6dc17eaad6" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
W20260709 12:22:17.409377 29944 leader_election.cc:343] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.423360 30218 leader_election.cc:290] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.423964 30019 leader_election.cc:304] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8; no voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:17.424899 30176 raft_consensus.cc:3060] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.426775 30128 raft_consensus.cc:2468] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.427074 30212 raft_consensus.cc:515] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.429131 30212 leader_election.cc:290] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.424475 29944 leader_election.cc:304] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: 
I20260709 12:22:17.432683 30054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.433512 30054 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.436578 30053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.436822 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.437369 30129 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.436671 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.438449 30128 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.439098 29944 leader_election.cc:304] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.435138 30212 raft_consensus.cc:2804] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:17.441606 30178 raft_consensus.cc:515] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.441078 29876 catalog_manager.cc:5697] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 0 to 1, leader changed from <none> to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.444892 30176 raft_consensus.cc:2749] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:17.437798 29877 catalog_manager.cc:5697] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "f7280ec9528e47eabc07fe33f45895d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:17.445802 30129 raft_consensus.cc:2468] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.446806 30171 tablet_bootstrap.cc:492] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:17.447324 30171 ts_tablet_manager.cc:1403] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.089s	user 0.019s	sys 0.007s
I20260709 12:22:17.446758 29877 catalog_manager.cc:5697] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "f7280ec9528e47eabc07fe33f45895d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:17.450335 30171 raft_consensus.cc:359] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.450620 30128 raft_consensus.cc:2468] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.450714 30178 leader_election.cc:290] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.450976 30171 raft_consensus.cc:385] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.451275 30171 raft_consensus.cc:740] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.451718 30055 raft_consensus.cc:2468] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
W20260709 12:22:17.442740 29944 leader_election.cc:343] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:17.451918 30171 consensus_queue.cc:260] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.441448 30218 raft_consensus.cc:2804] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.453131 30218 raft_consensus.cc:697] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.453713 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.442431 30212 raft_consensus.cc:493] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.455363 30054 raft_consensus.cc:2468] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.453931 30218 consensus_queue.cc:237] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.454493 30055 raft_consensus.cc:3060] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.456137 29944 leader_election.cc:304] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:17.455432 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.458804 30212 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.461174 30171 ts_tablet_manager.cc:1434] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 12:22:17.462023 30171 tablet_bootstrap.cc:492] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8: Bootstrap starting.
I20260709 12:22:17.462622 30178 raft_consensus.cc:2804] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.463299 30178 raft_consensus.cc:697] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.463204 29944 leader_election.cc:304] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.460475 30128 raft_consensus.cc:3060] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.465077 30213 raft_consensus.cc:2804] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.464148 30178 consensus_queue.cc:237] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.473117 30128 raft_consensus.cc:2468] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.469869 30212 raft_consensus.cc:515] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.478035 30212 leader_election.cc:290] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.475865 30213 raft_consensus.cc:697] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.484309 30213 consensus_queue.cc:237] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.487430 29944 leader_election.cc:304] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:17.488397 30212 raft_consensus.cc:2804] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.489187 30212 raft_consensus.cc:697] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.489252 30054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.489935 30054 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.490350 30212 consensus_queue.cc:237] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.490952 30055 raft_consensus.cc:2468] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.493263 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.493799 30128 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.496667 30054 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.501281 30171 tablet_bootstrap.cc:654] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:17.506529 30213 raft_consensus.cc:493] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.506572 29944 leader_election.cc:304] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: 
I20260709 12:22:17.507303 30213 raft_consensus.cc:515] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.510897 30218 raft_consensus.cc:2804] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:17.511466 30218 raft_consensus.cc:697] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.512081 30128 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 1.
I20260709 12:22:17.512295 30218 consensus_queue.cc:237] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.514808 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.512800 30213 leader_election.cc:290] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.509608 29876 catalog_manager.cc:5697] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 0 to 1, leader changed from <none> to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.514288 30054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8" is_pre_election: true
I20260709 12:22:17.517088 30171 tablet_bootstrap.cc:492] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8: No bootstrap required, opened a new log
I20260709 12:22:17.517513 30054 raft_consensus.cc:2393] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd46e724d7564358bf7e8d707dccab8e in current term 1: Already voted for candidate f7280ec9528e47eabc07fe33f45895d7 in this term.
I20260709 12:22:17.517503 30171 ts_tablet_manager.cc:1403] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8: Time spent bootstrapping tablet: real 0.056s	user 0.011s	sys 0.003s
I20260709 12:22:17.519176 29876 catalog_manager.cc:5697] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 0 to 1, leader changed from <none> to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.520269 30171 raft_consensus.cc:359] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.520905 30171 raft_consensus.cc:385] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:17.521174 30171 raft_consensus.cc:740] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Initialized, Role: FOLLOWER
I20260709 12:22:17.521821 30171 consensus_queue.cc:260] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.526870 30171 ts_tablet_manager.cc:1434] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:22:17.528825 29944 leader_election.cc:304] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e; no voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:17.529621 30212 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.536267 30212 raft_consensus.cc:2749] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:17.543243 29875 catalog_manager.cc:5697] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 0 to 1, leader changed from <none> to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.545037 29875 catalog_manager.cc:5697] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 0 to 1, leader changed from <none> to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.546617 29875 catalog_manager.cc:5697] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 0 to 1, leader changed from <none> to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.650825 30129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cda325da7a94860a205195db11e9949"
dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
mode: GRACEFUL
new_leader_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:35726
I20260709 12:22:17.651397 30129 raft_consensus.cc:606] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Received request to transfer leadership to TS bd46e724d7564358bf7e8d707dccab8e
I20260709 12:22:17.659198 30129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650"
dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
mode: GRACEFUL
new_leader_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:35726
I20260709 12:22:17.659677 30129 raft_consensus.cc:606] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Received request to transfer leadership to TS bd46e724d7564358bf7e8d707dccab8e
I20260709 12:22:17.673497 30204 consensus_queue.cc:1048] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.683809 29980 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:52410
I20260709 12:22:17.684396 29980 raft_consensus.cc:606] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:17.691054 30204 consensus_queue.cc:1048] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.694283 29980 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:52410
I20260709 12:22:17.695636 29980 raft_consensus.cc:606] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:17.702315 29980 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:52410
I20260709 12:22:17.702898 29980 raft_consensus.cc:606] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:17.704968 30223 raft_consensus.cc:493] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.705471 30223 raft_consensus.cc:515] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.707659 30223 leader_election.cc:290] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.709350 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.709383 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:17.710279 30129 raft_consensus.cc:2393] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5875c6c1c8024f41b05b9c8965c184f8 in current term 1: Already voted for candidate bd46e724d7564358bf7e8d707dccab8e in this term.
I20260709 12:22:17.711925 30019 leader_election.cc:304] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8; no voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:17.712824 30223 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.713028 29979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb6e17441f6747c0a655166876508d7f"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:52410
I20260709 12:22:17.713641 29979 raft_consensus.cc:606] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:17.720041 30223 raft_consensus.cc:2749] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:17.752027 30223 raft_consensus.cc:493] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.752558 30223 raft_consensus.cc:515] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.754772 30223 leader_election.cc:290] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.755499 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd46e724d7564358bf7e8d707dccab8e" is_pre_election: true
I20260709 12:22:17.755813 30129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7280ec9528e47eabc07fe33f45895d7" is_pre_election: true
I20260709 12:22:17.756559 30129 raft_consensus.cc:2393] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5875c6c1c8024f41b05b9c8965c184f8 in current term 1: Already voted for candidate bd46e724d7564358bf7e8d707dccab8e in this term.
I20260709 12:22:17.757447 30019 leader_election.cc:304] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8; no voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:17.758052 30223 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:17.764523 30223 raft_consensus.cc:2749] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:17.791783 30204 consensus_queue.cc:1048] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.797979 30204 raft_consensus.cc:993] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7: : Instructing follower bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:17.798450 30204 raft_consensus.cc:1081] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Signalling peer bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:17.801074 29980 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cda325da7a94860a205195db11e9949"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:52396
I20260709 12:22:17.801671 29980 raft_consensus.cc:493] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.802031 29980 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.804391 30204 consensus_queue.cc:1048] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.809646 29980 raft_consensus.cc:515] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.813457 29979 raft_consensus.cc:1240] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Rejecting Update request from peer f7280ec9528e47eabc07fe33f45895d7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:17.813867 30055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.814244 29980 leader_election.cc:290] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 2 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.815840 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.816422 30128 raft_consensus.cc:3055] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.816771 30128 raft_consensus.cc:740] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:17.816671 30212 consensus_queue.cc:1048] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.817679 30128 consensus_queue.cc:260] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.820056 30204 consensus_queue.cc:1059] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.824903 30128 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.827121 30184 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.835556 30184 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:17.833483 30128 raft_consensus.cc:2468] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 2.
I20260709 12:22:17.837916 29944 leader_election.cc:304] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:17.839121 30212 raft_consensus.cc:2804] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.839615 30212 raft_consensus.cc:697] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.840476 30212 consensus_queue.cc:237] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.859310 29876 catalog_manager.cc:5697] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 1 to 2, leader changed from f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67) to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.875409 30178 consensus_queue.cc:1048] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.879032 30212 consensus_queue.cc:1048] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.882706 30204 raft_consensus.cc:993] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7: : Instructing follower bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:17.883293 30204 raft_consensus.cc:1081] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Signalling peer bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:17.885646 29979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:52396
I20260709 12:22:17.886178 29979 raft_consensus.cc:493] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.886636 29979 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.893500 29979 raft_consensus.cc:515] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.894574 29980 raft_consensus.cc:1240] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Rejecting Update request from peer f7280ec9528e47eabc07fe33f45895d7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:17.896724 30208 consensus_queue.cc:1059] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.897487 30208 raft_consensus.cc:3055] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.897827 30208 raft_consensus.cc:740] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:17.898664 30208 consensus_queue.cc:260] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.899799 30208 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.910327 30053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:17.910931 30053 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.915961 30224 consensus_queue.cc:1048] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:17.917605 30224 raft_consensus.cc:993] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:17.919988 30224 raft_consensus.cc:1081] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:17.924322 30055 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55798
I20260709 12:22:17.925055 30055 raft_consensus.cc:493] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.925745 30053 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 2.
I20260709 12:22:17.925525 30055 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.926882 29944 leader_election.cc:304] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: 
I20260709 12:22:17.927695 30224 raft_consensus.cc:2804] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:17.928150 30224 raft_consensus.cc:697] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.933132 30178 consensus_queue.cc:1048] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.932281 30224 consensus_queue.cc:237] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.942839 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.936669 30055 raft_consensus.cc:515] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.947965 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:17.948731 29980 raft_consensus.cc:3055] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.949150 29980 raft_consensus.cc:740] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:17.950035 30128 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 2.
I20260709 12:22:17.953341 29979 leader_election.cc:290] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 2 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.955065 29877 catalog_manager.cc:5697] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 1 to 2, leader changed from f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67) to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:17.958820 30178 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.970979 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:17.971737 30128 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.950325 29980 consensus_queue.cc:260] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.974725 30212 raft_consensus.cc:993] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:17.975183 30212 raft_consensus.cc:1081] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:17.975219 29980 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.978852 30051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55798
I20260709 12:22:17.979573 30051 raft_consensus.cc:493] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:17.980039 30051 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:17.986869 30178 consensus_queue.cc:1048] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.988139 30178 consensus_queue.cc:1048] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.989872 30051 raft_consensus.cc:515] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:17.978330 30055 leader_election.cc:290] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.992336 30053 raft_consensus.cc:1240] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Rejecting Update request from peer bd46e724d7564358bf7e8d707dccab8e for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:17.998545 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:17.999140 29979 raft_consensus.cc:3055] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:17.999224 30051 leader_election.cc:290] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:17.999486 30126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:18.002954 30178 consensus_queue.cc:1059] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:17.999522 29979 raft_consensus.cc:740] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:18.006538 29979 consensus_queue.cc:260] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:18.007506 30128 raft_consensus.cc:2468] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:18.007954 30178 consensus_queue.cc:1048] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:18.008270 29980 raft_consensus.cc:2468] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:18.009213 30019 leader_election.cc:304] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:18.011078 30223 raft_consensus.cc:2804] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.015955 30224 consensus_queue.cc:1048] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:18.022485 29979 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.046468 30223 raft_consensus.cc:697] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:18.047420 30223 consensus_queue.cc:237] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
W20260709 12:22:18.051491 30245 log.cc:927] Time spent T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:22:18.051615 30218 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:18.053464 30218 consensus_queue.cc:1048] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:18.055876 29877 catalog_manager.cc:5697] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 1 to 2, leader changed from bd46e724d7564358bf7e8d707dccab8e (127.29.37.65) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.057973 29979 raft_consensus.cc:2468] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
W20260709 12:22:18.058876 30213 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:22:18.064095 30019 leader_election.cc:304] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:18.064940 30223 raft_consensus.cc:2804] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.065573 30223 raft_consensus.cc:697] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:18.066644 30223 consensus_queue.cc:237] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:18.074952 30218 raft_consensus.cc:993] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:18.075482 30218 raft_consensus.cc:1081] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:18.079290 30053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55798
I20260709 12:22:18.079916 30053 raft_consensus.cc:493] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.080293 30053 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.086199 30053 raft_consensus.cc:515] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:18.091008 30053 leader_election.cc:290] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:18.095860 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:18.096448 29979 raft_consensus.cc:3055] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.096604 30127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:18.096820 29979 raft_consensus.cc:740] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:18.097198 30127 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.097613 29979 consensus_queue.cc:260] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:18.098779 29979 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.102799 30224 raft_consensus.cc:993] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:18.103238 30224 raft_consensus.cc:1081] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:18.104378 30127 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:18.105388 30053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb6e17441f6747c0a655166876508d7f"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55798
I20260709 12:22:18.105916 30053 raft_consensus.cc:493] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.106302 30053 raft_consensus.cc:3060] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.107089 29979 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:18.107757 30019 leader_election.cc:304] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:18.108711 30249 raft_consensus.cc:2804] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.109232 30249 raft_consensus.cc:697] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:18.110105 30249 consensus_queue.cc:237] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:18.115268 30053 raft_consensus.cc:515] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:18.117822 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e17441f6747c0a655166876508d7f" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:18.118420 29979 raft_consensus.cc:3055] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:18.118746 29979 raft_consensus.cc:740] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:18.119508 29979 consensus_queue.cc:260] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:18.120553 29979 raft_consensus.cc:3060] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.121635 30127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb6e17441f6747c0a655166876508d7f" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:18.122179 30127 raft_consensus.cc:3060] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.124007 29875 catalog_manager.cc:5697] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 1 to 2, leader changed from bd46e724d7564358bf7e8d707dccab8e (127.29.37.65) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.126911 29979 raft_consensus.cc:2468] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:18.126263 29875 catalog_manager.cc:5697] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 1 to 2, leader changed from bd46e724d7564358bf7e8d707dccab8e (127.29.37.65) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.128032 30019 leader_election.cc:304] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: 
I20260709 12:22:18.128870 30249 raft_consensus.cc:2804] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:18.129225 30053 leader_election.cc:290] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:18.129390 30249 raft_consensus.cc:697] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:18.128710 30127 raft_consensus.cc:2468] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:18.130259 30249 consensus_queue.cc:237] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:18.141395 29875 catalog_manager.cc:5697] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 1 to 2, leader changed from bd46e724d7564358bf7e8d707dccab8e (127.29.37.65) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.322460 30053 raft_consensus.cc:1275] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.323621 30178 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.332512 30127 raft_consensus.cc:1275] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.333925 30224 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.404036 30053 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.409076 30053 raft_consensus.cc:1275] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.410339 30127 raft_consensus.cc:1275] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.410534 30224 consensus_queue.cc:1048] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.411633 30213 consensus_queue.cc:1048] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.509354 30127 raft_consensus.cc:1275] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.510603 30249 consensus_queue.cc:1048] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.517284 29979 raft_consensus.cc:1275] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.519055 30223 consensus_queue.cc:1048] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.546392 30127 raft_consensus.cc:1275] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.547435 30223 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.554661 29979 raft_consensus.cc:1275] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.555891 30252 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.600693 29979 raft_consensus.cc:1275] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.601848 30252 consensus_queue.cc:1048] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.610559 30127 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:18.616593 30127 raft_consensus.cc:1275] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.617869 30252 consensus_queue.cc:1048] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.690728 29979 raft_consensus.cc:1275] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.691993 30252 consensus_queue.cc:1048] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:18.699097 30127 raft_consensus.cc:1275] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:18.700714 30252 consensus_queue.cc:1048] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:20.746430 29845 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:22:20.747169 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:20.766920 29979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
mode: GRACEFUL
new_leader_uuid: "f7280ec9528e47eabc07fe33f45895d7"
 from {username='slave'} at 127.0.0.1:52416
I20260709 12:22:20.767555 29979 raft_consensus.cc:606] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Received request to transfer leadership to TS f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:20.777944 30053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
mode: GRACEFUL
new_leader_uuid: "f7280ec9528e47eabc07fe33f45895d7"
 from {username='slave'} at 127.0.0.1:55808
I20260709 12:22:20.778445 30053 raft_consensus.cc:606] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Received request to transfer leadership to TS f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:20.779493 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:20.779942 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:20.850550 30262 raft_consensus.cc:993] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8: : Instructing follower f7280ec9528e47eabc07fe33f45895d7 to start an election
I20260709 12:22:20.851004 30252 raft_consensus.cc:1081] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Signalling peer f7280ec9528e47eabc07fe33f45895d7 to start an election
I20260709 12:22:20.852378 30128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82"
dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:22:20.852773 30128 raft_consensus.cc:493] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:20.853011 30128 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.857455 30128 raft_consensus.cc:515] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:20.859416 30128 leader_election.cc:290] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 3 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:20.860008 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:20.860417 30053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:20.860574 29979 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.861042 30053 raft_consensus.cc:3055] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:20.861315 30053 raft_consensus.cc:740] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:20.861990 30053 consensus_queue.cc:260] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:20.862890 30053 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.865803 29979 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 3.
I20260709 12:22:20.866904 30093 leader_election.cc:304] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:20.867542 30204 raft_consensus.cc:2804] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:20.867605 30053 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 3.
I20260709 12:22:20.868757 30204 raft_consensus.cc:697] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Becoming Leader. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:20.869565 30204 consensus_queue.cc:237] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:20.875458 29875 catalog_manager.cc:5697] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 reported cstate change: term changed from 2 to 3, leader changed from 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66) to f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67). New cstate: current_term: 3 leader_uuid: "f7280ec9528e47eabc07fe33f45895d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:20.925092 30258 raft_consensus.cc:993] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e: : Instructing follower f7280ec9528e47eabc07fe33f45895d7 to start an election
I20260709 12:22:20.925542 30224 raft_consensus.cc:1081] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Signalling peer f7280ec9528e47eabc07fe33f45895d7 to start an election
I20260709 12:22:20.926849 30128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650"
dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
 from {username='slave'} at 127.0.0.1:35720
I20260709 12:22:20.927291 30128 raft_consensus.cc:493] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:20.927582 30128 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.932039 30128 raft_consensus.cc:515] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:20.933593 30128 leader_election.cc:290] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 3 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:20.934332 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:20.934618 30053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:20.934932 29979 raft_consensus.cc:3055] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:20.935138 30053 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.935256 29979 raft_consensus.cc:740] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:20.936045 29979 consensus_queue.cc:260] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:20.937176 29979 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:20.941646 30053 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 3.
I20260709 12:22:20.942689 30093 leader_election.cc:304] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:20.943214 29979 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 3.
I20260709 12:22:20.943341 30204 raft_consensus.cc:2804] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:20.943862 30204 raft_consensus.cc:697] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Becoming Leader. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:20.944654 30204 consensus_queue.cc:237] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:20.951122 29875 catalog_manager.cc:5697] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 reported cstate change: term changed from 2 to 3, leader changed from bd46e724d7564358bf7e8d707dccab8e (127.29.37.65) to f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67). New cstate: current_term: 3 leader_uuid: "f7280ec9528e47eabc07fe33f45895d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:21.324944 30053 raft_consensus.cc:1275] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Refusing update from remote peer f7280ec9528e47eabc07fe33f45895d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:21.326077 30204 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.336228 29979 raft_consensus.cc:1275] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Refusing update from remote peer f7280ec9528e47eabc07fe33f45895d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:21.338932 30204 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.384917 29979 raft_consensus.cc:1275] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Refusing update from remote peer f7280ec9528e47eabc07fe33f45895d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:21.386286 30204 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.387703 30053 raft_consensus.cc:1275] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Refusing update from remote peer f7280ec9528e47eabc07fe33f45895d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:21.389024 30204 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:21.780778 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.786729 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:21.787102 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:22.787868 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:22.792695 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:22.793066 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:23.793876 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:23.799106 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:23.799510 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:24.800675 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:24.810211 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:24.810889 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:25.812077 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:25.817070 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:25.817516 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:26.818639 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:26.823691 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:26.824059 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:27.825101 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:27.830672 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:27.831044 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:28.831935 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:28.836854 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:28.837284 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:29.838155 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:29.843075 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:29.843484 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:30.870425 29845 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:22:30.874361 29979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cda325da7a94860a205195db11e9949"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:52410
I20260709 12:22:30.874949 29979 raft_consensus.cc:606] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:30.879307 29979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:52410
I20260709 12:22:30.879703 29979 raft_consensus.cc:606] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:30.883800 30128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650"
dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:35726
I20260709 12:22:30.884150 30128 raft_consensus.cc:606] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:30.888106 29979 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:52410
I20260709 12:22:30.888453 29979 raft_consensus.cc:606] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:30.892221 30128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82"
dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
mode: GRACEFUL
new_leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:35726
I20260709 12:22:30.892555 30128 raft_consensus.cc:606] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Received request to transfer leadership to TS 5875c6c1c8024f41b05b9c8965c184f8
I20260709 12:22:30.934060 30290 raft_consensus.cc:993] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:30.934617 30280 raft_consensus.cc:1081] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:30.936024 30051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55794
I20260709 12:22:30.936498 30051 raft_consensus.cc:493] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:30.936829 30051 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:30.941752 30051 raft_consensus.cc:515] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:30.943475 30051 leader_election.cc:290] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 4 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:30.944185 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:30.944466 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:30.944945 29979 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:30.945009 30128 raft_consensus.cc:3055] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:30.945363 30128 raft_consensus.cc:740] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:30.946154 30128 consensus_queue.cc:260] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:30.947077 30128 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:30.949023 29979 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 4.
I20260709 12:22:30.950011 30019 leader_election.cc:304] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: 
I20260709 12:22:30.950824 30252 raft_consensus.cc:2804] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:30.951238 30128 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 4.
I20260709 12:22:30.951825 30252 raft_consensus.cc:697] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:30.952775 30252 consensus_queue.cc:237] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:30.959183 29876 catalog_manager.cc:5697] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 3 to 4, leader changed from f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 4 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.149243 30289 raft_consensus.cc:993] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:31.149760 30297 raft_consensus.cc:1081] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:31.151072 30051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55798
I20260709 12:22:31.151530 30051 raft_consensus.cc:493] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.151854 30051 raft_consensus.cc:3060] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.154857 30289 raft_consensus.cc:993] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:31.155337 30297 raft_consensus.cc:1081] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:31.156682 30053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cda325da7a94860a205195db11e9949"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55798
I20260709 12:22:31.157075 30053 raft_consensus.cc:493] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.157339 30053 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.158025 30051 raft_consensus.cc:515] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.159725 30051 leader_election.cc:290] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:31.160231 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:31.160732 29979 raft_consensus.cc:3055] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:31.160990 29979 raft_consensus.cc:740] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:31.160974 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756a83b7a17a461d9ff9771f3f411793" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:31.161623 30128 raft_consensus.cc:3060] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.161828 29979 consensus_queue.cc:260] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.162194 30053 raft_consensus.cc:515] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.162959 29979 raft_consensus.cc:3060] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.163856 30053 leader_election.cc:290] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 3 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:31.164621 29980 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:31.165201 29980 raft_consensus.cc:3055] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:31.165445 30127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cda325da7a94860a205195db11e9949" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:31.166039 30127 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.167254 30128 raft_consensus.cc:2468] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:31.165545 29980 raft_consensus.cc:740] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:31.168222 30019 leader_election.cc:304] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:31.168326 29980 consensus_queue.cc:260] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.169114 30291 raft_consensus.cc:2804] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.169390 29980 raft_consensus.cc:3060] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:31.169592 30291 raft_consensus.cc:697] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:31.170490 30291 consensus_queue.cc:237] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.173018 30127 raft_consensus.cc:2468] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 3.
I20260709 12:22:31.174234 30019 leader_election.cc:304] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:31.174971 30252 raft_consensus.cc:2804] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:31.176286 30252 raft_consensus.cc:697] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:31.176487 29980 raft_consensus.cc:2468] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 3.
I20260709 12:22:31.174777 29979 raft_consensus.cc:2468] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:31.177126 30252 consensus_queue.cc:237] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.182471 29876 catalog_manager.cc:5697] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 1 to 2, leader changed from bd46e724d7564358bf7e8d707dccab8e (127.29.37.65) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.191877 29876 catalog_manager.cc:5697] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 2 to 3, leader changed from bd46e724d7564358bf7e8d707dccab8e (127.29.37.65) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 3 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.256278 30289 raft_consensus.cc:993] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:31.256712 30297 raft_consensus.cc:1081] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:31.257915 30053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55798
I20260709 12:22:31.258358 30053 raft_consensus.cc:493] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.258622 30053 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.262210 30053 raft_consensus.cc:515] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.263511 30053 leader_election.cc:290] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:31.264217 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:31.264526 30127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:31.264719 29979 raft_consensus.cc:3055] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:31.265023 29979 raft_consensus.cc:740] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:31.265059 30127 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.265679 29979 consensus_queue.cc:260] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.266517 29979 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:31.269299 30127 raft_consensus.cc:2468] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:31.270128 30019 leader_election.cc:304] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:31.270534 29979 raft_consensus.cc:2468] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 2.
I20260709 12:22:31.270766 30252 raft_consensus.cc:2804] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:31.271188 30252 raft_consensus.cc:697] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:31.271919 30252 consensus_queue.cc:237] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.277799 29876 catalog_manager.cc:5697] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 1 to 2, leader changed from bd46e724d7564358bf7e8d707dccab8e (127.29.37.65) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.483516 30280 raft_consensus.cc:993] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7: : Instructing follower 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:31.483943 30302 raft_consensus.cc:1081] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Signalling peer 5875c6c1c8024f41b05b9c8965c184f8 to start an election
I20260709 12:22:31.485198 30053 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
 from {username='slave'} at 127.0.0.1:55794
I20260709 12:22:31.485561 30053 raft_consensus.cc:493] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:31.485785 30053 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.489415 30053 raft_consensus.cc:515] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.490622 30053 leader_election.cc:290] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 4 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:31.491377 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:31.491796 30127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "5875c6c1c8024f41b05b9c8965c184f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:31.491959 29979 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.492370 30127 raft_consensus.cc:3055] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:31.492802 30127 raft_consensus.cc:740] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:31.493700 30127 consensus_queue.cc:260] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.494863 30127 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:31.498361 29979 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 4.
I20260709 12:22:31.499281 30019 leader_election.cc:304] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5875c6c1c8024f41b05b9c8965c184f8, bd46e724d7564358bf7e8d707dccab8e; no voters: 
I20260709 12:22:31.499871 30252 raft_consensus.cc:2804] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:31.500224 30252 raft_consensus.cc:697] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Becoming Leader. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:31.500738 30127 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5875c6c1c8024f41b05b9c8965c184f8 in term 4.
I20260709 12:22:31.500860 30252 consensus_queue.cc:237] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:31.508461 29876 catalog_manager.cc:5697] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 reported cstate change: term changed from 3 to 4, leader changed from f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67) to 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66). New cstate: current_term: 4 leader_uuid: "5875c6c1c8024f41b05b9c8965c184f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.513374 30127 raft_consensus.cc:1275] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 4 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:31.514554 30252 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:31.526749 29979 raft_consensus.cc:1275] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 4 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:31.528275 30252 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:31.590346 29979 raft_consensus.cc:1275] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.591401 30252 consensus_queue.cc:1048] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:31.600684 30127 raft_consensus.cc:1275] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:31.604338 30299 consensus_queue.cc:1048] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:31.609787 29980 raft_consensus.cc:1275] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:31.611024 30299 consensus_queue.cc:1048] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:31.619647 30127 raft_consensus.cc:1275] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:31.621946 30310 consensus_queue.cc:1048] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:31.793988 29980 raft_consensus.cc:1275] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:31.795111 30310 consensus_queue.cc:1048] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:31.803555 30127 raft_consensus.cc:1275] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:31.805503 30299 consensus_queue.cc:1048] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:31.883914 29980 raft_consensus.cc:1275] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 4 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:31.885036 30299 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:31.893554 30127 raft_consensus.cc:1275] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 4 FOLLOWER]: Refusing update from remote peer 5875c6c1c8024f41b05b9c8965c184f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:31.895210 30299 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:33.931070 29845 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:22:33.931794 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:33.938618 30053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
mode: GRACEFUL
new_leader_uuid: "f7280ec9528e47eabc07fe33f45895d7"
 from {username='slave'} at 127.0.0.1:55808
I20260709 12:22:33.939131 30053 raft_consensus.cc:606] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Received request to transfer leadership to TS f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:33.941104 30053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
mode: GRACEFUL
new_leader_uuid: "f7280ec9528e47eabc07fe33f45895d7"
 from {username='slave'} at 127.0.0.1:55808
I20260709 12:22:33.941501 30053 raft_consensus.cc:606] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Received request to transfer leadership to TS f7280ec9528e47eabc07fe33f45895d7
I20260709 12:22:33.943277 30053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
mode: GRACEFUL
new_leader_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:55808
I20260709 12:22:33.943670 30053 raft_consensus.cc:606] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Received request to transfer leadership to TS bd46e724d7564358bf7e8d707dccab8e
I20260709 12:22:33.945399 30053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
mode: GRACEFUL
new_leader_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:55808
I20260709 12:22:33.945782 30053 raft_consensus.cc:606] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Received request to transfer leadership to TS bd46e724d7564358bf7e8d707dccab8e
I20260709 12:22:33.947512 30053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd"
dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
mode: GRACEFUL
new_leader_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:55808
I20260709 12:22:33.947896 30053 raft_consensus.cc:606] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Received request to transfer leadership to TS bd46e724d7564358bf7e8d707dccab8e
I20260709 12:22:33.948729 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:33.949259 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:33.951189 30318 raft_consensus.cc:993] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8: : Instructing follower f7280ec9528e47eabc07fe33f45895d7 to start an election
I20260709 12:22:33.951581 30310 raft_consensus.cc:1081] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Signalling peer f7280ec9528e47eabc07fe33f45895d7 to start an election
I20260709 12:22:33.952744 30128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650"
dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:22:33.953122 30128 raft_consensus.cc:493] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:33.953387 30128 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.957267 30128 raft_consensus.cc:515] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:33.959178 30128 leader_election.cc:290] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 5 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:33.959668 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:33.960045 30053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03abc00bdfec46bd817f6b535c1b4650" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:33.960181 29979 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.960506 30053 raft_consensus.cc:3055] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:33.960772 30053 raft_consensus.cc:740] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:33.961370 30053 consensus_queue.cc:260] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:33.962181 30053 raft_consensus.cc:3060] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:33.964775 29979 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 5.
I20260709 12:22:33.965821 30093 leader_election.cc:304] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:33.966665 30302 raft_consensus.cc:2804] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:33.967491 30053 raft_consensus.cc:2468] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 5.
I20260709 12:22:33.967672 30302 raft_consensus.cc:697] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 5 LEADER]: Becoming Leader. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:33.968538 30302 consensus_queue.cc:237] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:33.972628 30318 raft_consensus.cc:993] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8: : Instructing follower bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:33.973032 30310 raft_consensus.cc:1081] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Signalling peer bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:33.974426 29979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:52388
I20260709 12:22:33.974810 29979 raft_consensus.cc:493] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:33.975132 29979 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:33.975641 29875 catalog_manager.cc:5697] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 reported cstate change: term changed from 4 to 5, leader changed from 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66) to f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67). New cstate: current_term: 5 leader_uuid: "f7280ec9528e47eabc07fe33f45895d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.980573 29979 raft_consensus.cc:515] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:33.982863 29979 leader_election.cc:290] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 3 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:33.983588 30053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:33.983721 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d6f8015ea804df0bd92ac6cfb257d29" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:33.984184 30053 raft_consensus.cc:3055] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:33.984302 30128 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:33.984642 30053 raft_consensus.cc:740] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:33.985288 30053 consensus_queue.cc:260] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:33.986074 30053 raft_consensus.cc:3060] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:33.990731 30128 raft_consensus.cc:2468] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 3.
I20260709 12:22:33.991039 30053 raft_consensus.cc:2468] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 3.
I20260709 12:22:33.991905 29944 leader_election.cc:304] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:33.993403 30325 raft_consensus.cc:2804] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:33.994583 30325 raft_consensus.cc:697] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 3 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:33.995386 30325 consensus_queue.cc:237] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.001518 29875 catalog_manager.cc:5697] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 2 to 3, leader changed from 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66) to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 3 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.164225 30318 raft_consensus.cc:993] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8: : Instructing follower bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:34.164669 30310 raft_consensus.cc:1081] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Signalling peer bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:34.165956 29979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:52388
I20260709 12:22:34.166388 29979 raft_consensus.cc:493] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.166667 29979 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.170948 29979 raft_consensus.cc:515] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.172339 29979 leader_election.cc:290] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 3 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:34.172917 30053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:34.173215 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6e1c832aab64224b83c0aaa7dab64fd" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:34.173437 30053 raft_consensus.cc:3055] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.173774 30128 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.173780 30053 raft_consensus.cc:740] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:34.174772 30053 consensus_queue.cc:260] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.175637 30053 raft_consensus.cc:3060] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.176367 30318 raft_consensus.cc:993] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8: : Instructing follower bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:34.176821 30310 raft_consensus.cc:1081] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Signalling peer bd46e724d7564358bf7e8d707dccab8e to start an election
I20260709 12:22:34.178287 29979 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852"
dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
 from {username='slave'} at 127.0.0.1:52388
I20260709 12:22:34.178735 29979 raft_consensus.cc:493] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.179068 29979 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.179111 30128 raft_consensus.cc:2468] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 3.
I20260709 12:22:34.180187 29944 leader_election.cc:304] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:34.180893 30325 raft_consensus.cc:2804] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:34.181063 30053 raft_consensus.cc:2468] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 3.
I20260709 12:22:34.181376 30325 raft_consensus.cc:697] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 3 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:34.182344 30325 consensus_queue.cc:237] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.184216 29979 raft_consensus.cc:515] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.186707 30053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:34.186847 29979 leader_election.cc:290] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 3 election: Requested vote from peers 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929), f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67:43943)
I20260709 12:22:34.187192 30053 raft_consensus.cc:3055] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:34.187475 30053 raft_consensus.cc:740] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:34.188335 30128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f0adea973d4d23918b329c2b88c852" candidate_uuid: "bd46e724d7564358bf7e8d707dccab8e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
I20260709 12:22:34.188162 30053 consensus_queue.cc:260] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.188786 30128 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.189145 30053 raft_consensus.cc:3060] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:34.189996 29875 catalog_manager.cc:5697] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 2 to 3, leader changed from 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66) to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 3 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.192489 30128 raft_consensus.cc:2468] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 3.
I20260709 12:22:34.193378 29944 leader_election.cc:304] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:34.194046 30325 raft_consensus.cc:2804] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:34.194077 30053 raft_consensus.cc:2468] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd46e724d7564358bf7e8d707dccab8e in term 3.
I20260709 12:22:34.194717 30325 raft_consensus.cc:697] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 3 LEADER]: Becoming Leader. State: Replica: bd46e724d7564358bf7e8d707dccab8e, State: Running, Role: LEADER
I20260709 12:22:34.195461 30325 consensus_queue.cc:237] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.201455 29875 catalog_manager.cc:5697] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e reported cstate change: term changed from 2 to 3, leader changed from 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66) to bd46e724d7564358bf7e8d707dccab8e (127.29.37.65). New cstate: current_term: 3 leader_uuid: "bd46e724d7564358bf7e8d707dccab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.367568 29979 raft_consensus.cc:1275] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 5 FOLLOWER]: Refusing update from remote peer f7280ec9528e47eabc07fe33f45895d7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:34.369707 30302 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:34.374435 30318 raft_consensus.cc:993] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8: : Instructing follower f7280ec9528e47eabc07fe33f45895d7 to start an election
I20260709 12:22:34.375511 30310 raft_consensus.cc:1081] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Signalling peer f7280ec9528e47eabc07fe33f45895d7 to start an election
I20260709 12:22:34.377867 30128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82"
dest_uuid: "f7280ec9528e47eabc07fe33f45895d7"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:22:34.378774 30128 raft_consensus.cc:493] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.379211 30128 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.393851 30053 raft_consensus.cc:1275] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 5 FOLLOWER]: Refusing update from remote peer f7280ec9528e47eabc07fe33f45895d7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:34.395937 30319 consensus_queue.cc:1048] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:34.390120 30128 raft_consensus.cc:515] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.400170 30128 leader_election.cc:290] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 5 election: Requested vote from peers bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627), 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929)
I20260709 12:22:34.400484 29979 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "bd46e724d7564358bf7e8d707dccab8e"
I20260709 12:22:34.401266 29979 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.402495 30051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cd34ffceb744b67a451d4a03c5d5c82" candidate_uuid: "f7280ec9528e47eabc07fe33f45895d7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5875c6c1c8024f41b05b9c8965c184f8"
I20260709 12:22:34.403236 30051 raft_consensus.cc:3055] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:34.403534 30051 raft_consensus.cc:740] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5875c6c1c8024f41b05b9c8965c184f8, State: Running, Role: LEADER
I20260709 12:22:34.404763 30051 consensus_queue.cc:260] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.408212 30051 raft_consensus.cc:3060] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.413064 29979 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 5.
I20260709 12:22:34.414379 30093 leader_election.cc:304] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd46e724d7564358bf7e8d707dccab8e, f7280ec9528e47eabc07fe33f45895d7; no voters: 
I20260709 12:22:34.415357 30335 raft_consensus.cc:2804] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:34.416941 30335 raft_consensus.cc:697] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 5 LEADER]: Becoming Leader. State: Replica: f7280ec9528e47eabc07fe33f45895d7, State: Running, Role: LEADER
I20260709 12:22:34.415954 30051 raft_consensus.cc:2468] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7280ec9528e47eabc07fe33f45895d7 in term 5.
I20260709 12:22:34.417852 30335 consensus_queue.cc:237] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } }
I20260709 12:22:34.424528 29876 catalog_manager.cc:5697] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 reported cstate change: term changed from 4 to 5, leader changed from 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66) to f7280ec9528e47eabc07fe33f45895d7 (127.29.37.67). New cstate: current_term: 5 leader_uuid: "f7280ec9528e47eabc07fe33f45895d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:34.453572 30128 raft_consensus.cc:1275] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.454756 30325 consensus_queue.cc:1048] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.465678 30051 raft_consensus.cc:1275] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.467095 30340 consensus_queue.cc:1048] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.583067 30128 raft_consensus.cc:1275] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.584030 30340 consensus_queue.cc:1048] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.592702 30051 raft_consensus.cc:1275] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.594398 30326 consensus_queue.cc:1048] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:34.600188 30128 raft_consensus.cc:1275] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.601229 30326 consensus_queue.cc:1048] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7280ec9528e47eabc07fe33f45895d7" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.611361 30051 raft_consensus.cc:1275] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Refusing update from remote peer bd46e724d7564358bf7e8d707dccab8e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:34.612589 30326 consensus_queue.cc:1048] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:34.856696 30051 raft_consensus.cc:1275] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 5 FOLLOWER]: Refusing update from remote peer f7280ec9528e47eabc07fe33f45895d7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:34.857815 30335 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5875c6c1c8024f41b05b9c8965c184f8" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:34.866680 29979 raft_consensus.cc:1275] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 5 FOLLOWER]: Refusing update from remote peer f7280ec9528e47eabc07fe33f45895d7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:34.868350 30347 consensus_queue.cc:1048] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd46e724d7564358bf7e8d707dccab8e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:34.950601 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:34.955394 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:34.955780 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:35.957005 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:35.961663 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:35.962066 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:36.963035 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:36.967893 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:36.968294 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:37.969252 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:37.974078 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:37.974496 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:38.975502 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:38.980243 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:38.980618 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:39.981523 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:39.986141 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:39.986545 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:40.987422 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:40.992163 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:40.992532 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:41.993466 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:41.998046 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:41.998500 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:42.999467 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:43.004176 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:43.004546 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:44.030375 29845 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:22:44.030895 29845 tablet_server.cc:179] TabletServer@127.29.37.65:0 shutting down...
W20260709 12:22:44.039201 30093 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:22:44.044667 30093 consensus_peers.cc:597] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 -> Peer bd46e724d7564358bf7e8d707dccab8e (127.29.37.65:38627): Couldn't send request to peer bd46e724d7564358bf7e8d707dccab8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:44.058822 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:44.059541 29845 tablet_replica.cc:333] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.060173 29845 raft_consensus.cc:2243] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.060874 29845 raft_consensus.cc:2272] T 6d6f8015ea804df0bd92ac6cfb257d29 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.062942 29845 tablet_replica.cc:333] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.063436 29845 raft_consensus.cc:2243] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.063786 29845 raft_consensus.cc:2272] T 03abc00bdfec46bd817f6b535c1b4650 P bd46e724d7564358bf7e8d707dccab8e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.065138 29845 tablet_replica.cc:333] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.065559 29845 raft_consensus.cc:2243] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.066145 29845 raft_consensus.cc:2272] T 96f0adea973d4d23918b329c2b88c852 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.067655 29845 tablet_replica.cc:333] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.068078 29845 raft_consensus.cc:2243] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.068410 29845 raft_consensus.cc:2272] T 1cd34ffceb744b67a451d4a03c5d5c82 P bd46e724d7564358bf7e8d707dccab8e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.069726 29845 tablet_replica.cc:333] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.070153 29845 raft_consensus.cc:2243] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.070850 29845 raft_consensus.cc:2272] T f6e1c832aab64224b83c0aaa7dab64fd P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.072243 29845 tablet_replica.cc:333] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.072644 29845 raft_consensus.cc:2243] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.072954 29845 raft_consensus.cc:2272] T 1cda325da7a94860a205195db11e9949 P bd46e724d7564358bf7e8d707dccab8e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.074515 29845 tablet_replica.cc:333] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.074895 29845 raft_consensus.cc:2243] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.075214 29845 raft_consensus.cc:2272] T eb6e17441f6747c0a655166876508d7f P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.076611 29845 tablet_replica.cc:333] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.076982 29845 raft_consensus.cc:2243] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.077311 29845 raft_consensus.cc:2272] T 756a83b7a17a461d9ff9771f3f411793 P bd46e724d7564358bf7e8d707dccab8e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.078783 29845 tablet_replica.cc:333] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e: stopping tablet replica
I20260709 12:22:44.079208 29845 raft_consensus.cc:2243] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.079573 29845 raft_consensus.cc:2272] T 8c934a48601e4629823e2c6dc17eaad6 P bd46e724d7564358bf7e8d707dccab8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.162096 29845 tablet_server.cc:196] TabletServer@127.29.37.65:0 shutdown complete.
I20260709 12:22:44.179392 29845 tablet_server.cc:179] TabletServer@127.29.37.66:0 shutting down...
W20260709 12:22:44.200451 30093 consensus_peers.cc:597] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 -> Peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Couldn't send request to peer 5875c6c1c8024f41b05b9c8965c184f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:44.207646 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:44.208263 29845 tablet_replica.cc:333] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.208802 29845 raft_consensus.cc:2243] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.209252 29845 raft_consensus.cc:2272] T 6d6f8015ea804df0bd92ac6cfb257d29 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.210983 29845 tablet_replica.cc:333] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.211503 29845 raft_consensus.cc:2243] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.211906 29845 raft_consensus.cc:2272] T f6e1c832aab64224b83c0aaa7dab64fd P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.213341 29845 tablet_replica.cc:333] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.213820 29845 raft_consensus.cc:2243] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.214205 29845 raft_consensus.cc:2272] T 1cd34ffceb744b67a451d4a03c5d5c82 P 5875c6c1c8024f41b05b9c8965c184f8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.215718 29845 tablet_replica.cc:333] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.216184 29845 raft_consensus.cc:2243] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.216857 29845 raft_consensus.cc:2272] T eb6e17441f6747c0a655166876508d7f P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.218495 29845 tablet_replica.cc:333] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.218972 29845 raft_consensus.cc:2243] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.219363 29845 raft_consensus.cc:2272] T 03abc00bdfec46bd817f6b535c1b4650 P 5875c6c1c8024f41b05b9c8965c184f8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.220839 29845 tablet_replica.cc:333] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.221302 29845 raft_consensus.cc:2243] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.221671 29845 raft_consensus.cc:2272] T 96f0adea973d4d23918b329c2b88c852 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.223074 29845 tablet_replica.cc:333] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.223556 29845 raft_consensus.cc:2243] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.224223 29845 raft_consensus.cc:2272] T 1cda325da7a94860a205195db11e9949 P 5875c6c1c8024f41b05b9c8965c184f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.225675 29845 tablet_replica.cc:333] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.226136 29845 raft_consensus.cc:2243] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.226542 29845 raft_consensus.cc:2272] T 8c934a48601e4629823e2c6dc17eaad6 P 5875c6c1c8024f41b05b9c8965c184f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.227929 29845 tablet_replica.cc:333] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8: stopping tablet replica
I20260709 12:22:44.228333 29845 raft_consensus.cc:2243] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.228973 29845 raft_consensus.cc:2272] T 756a83b7a17a461d9ff9771f3f411793 P 5875c6c1c8024f41b05b9c8965c184f8 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:44.252355 30093 consensus_peers.cc:597] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 -> Peer 5875c6c1c8024f41b05b9c8965c184f8 (127.29.37.66:37929): Couldn't send request to peer 5875c6c1c8024f41b05b9c8965c184f8. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:37929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:44.254005 29845 tablet_server.cc:196] TabletServer@127.29.37.66:0 shutdown complete.
I20260709 12:22:44.269241 29845 tablet_server.cc:179] TabletServer@127.29.37.67:0 shutting down...
I20260709 12:22:44.295921 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:44.296492 29845 tablet_replica.cc:333] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.297043 29845 raft_consensus.cc:2243] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.297505 29845 raft_consensus.cc:2272] T eb6e17441f6747c0a655166876508d7f P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.299268 29845 tablet_replica.cc:333] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.299741 29845 raft_consensus.cc:2243] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.300137 29845 raft_consensus.cc:2272] T 6d6f8015ea804df0bd92ac6cfb257d29 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.301512 29845 tablet_replica.cc:333] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.301978 29845 raft_consensus.cc:2243] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.302407 29845 raft_consensus.cc:2272] T 1cda325da7a94860a205195db11e9949 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.303804 29845 tablet_replica.cc:333] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.304288 29845 raft_consensus.cc:2243] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.304956 29845 raft_consensus.cc:2272] T 03abc00bdfec46bd817f6b535c1b4650 P f7280ec9528e47eabc07fe33f45895d7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.306416 29845 tablet_replica.cc:333] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.306857 29845 raft_consensus.cc:2243] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.307240 29845 raft_consensus.cc:2272] T 96f0adea973d4d23918b329c2b88c852 P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.308596 29845 tablet_replica.cc:333] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.309052 29845 raft_consensus.cc:2243] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.309417 29845 raft_consensus.cc:2272] T 756a83b7a17a461d9ff9771f3f411793 P f7280ec9528e47eabc07fe33f45895d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.311018 29845 tablet_replica.cc:333] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.311443 29845 raft_consensus.cc:2243] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.312088 29845 raft_consensus.cc:2272] T 8c934a48601e4629823e2c6dc17eaad6 P f7280ec9528e47eabc07fe33f45895d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.313575 29845 tablet_replica.cc:333] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.314006 29845 raft_consensus.cc:2243] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.314690 29845 raft_consensus.cc:2272] T 1cd34ffceb744b67a451d4a03c5d5c82 P f7280ec9528e47eabc07fe33f45895d7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.316057 29845 tablet_replica.cc:333] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7: stopping tablet replica
I20260709 12:22:44.316481 29845 raft_consensus.cc:2243] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:44.316848 29845 raft_consensus.cc:2272] T f6e1c832aab64224b83c0aaa7dab64fd P f7280ec9528e47eabc07fe33f45895d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.343573 29845 tablet_server.cc:196] TabletServer@127.29.37.67:0 shutdown complete.
I20260709 12:22:44.364316 29845 master.cc:562] Master@127.29.37.126:34227 shutting down...
I20260709 12:22:44.385574 29845 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:44.386437 29845 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:44.386813 29845 tablet_replica.cc:333] T 00000000000000000000000000000000 P f6ec5e1f8fe34ba59c772c3141235a64: stopping tablet replica
I20260709 12:22:44.405788 29845 master.cc:584] Master@127.29.37.126:34227 shutdown complete.
5875c6c1c8024f41b05b9c8965c184f8  4
bd46e724d7564358bf7e8d707dccab8e  4
f7280ec9528e47eabc07fe33f45895d7  1
5875c6c1c8024f41b05b9c8965c184f8  3
bd46e724d7564358bf7e8d707dccab8e  3
f7280ec9528e47eabc07fe33f45895d7  3
5875c6c1c8024f41b05b9c8965c184f8  8
bd46e724d7564358bf7e8d707dccab8e  0
f7280ec9528e47eabc07fe33f45895d7  1
5875c6c1c8024f41b05b9c8965c184f8  3
bd46e724d7564358bf7e8d707dccab8e  3
f7280ec9528e47eabc07fe33f45895d7  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (30196 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:22:44.450073 29845 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.29.37.126:44267
I20260709 12:22:44.451470 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:44.457127 30357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.457455 30358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.458149 30360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:44.459215 29845 server_base.cc:1061] running on GCE node
I20260709 12:22:44.460129 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:44.460340 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:44.460498 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599764460483 us; error 0 us; skew 500 ppm
I20260709 12:22:44.461027 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:44.463344 29845 webserver.cc:533] Webserver started at http://127.29.37.126:43659/ using document root <none> and password file <none>
I20260709 12:22:44.463827 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:44.464022 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:44.464293 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:44.465445 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/master-0-root/instance:
uuid: "8175c3d1e5934060aacf4b050f8533bd"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-g1m6"
I20260709 12:22:44.469600 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:22:44.472692 30365 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.473510 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:22:44.473735 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/master-0-root
uuid: "8175c3d1e5934060aacf4b050f8533bd"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-g1m6"
I20260709 12:22:44.473959 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:44.483177 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:44.484231 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:44.520516 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.126:44267
I20260709 12:22:44.520601 30416 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.126:44267 every 8 connection(s)
I20260709 12:22:44.524518 30417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:44.533332 30417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd: Bootstrap starting.
I20260709 12:22:44.537564 30417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:44.541353 30417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd: No bootstrap required, opened a new log
I20260709 12:22:44.543313 30417 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8175c3d1e5934060aacf4b050f8533bd" member_type: VOTER }
I20260709 12:22:44.543704 30417 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:44.543875 30417 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8175c3d1e5934060aacf4b050f8533bd, State: Initialized, Role: FOLLOWER
I20260709 12:22:44.544353 30417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8175c3d1e5934060aacf4b050f8533bd" member_type: VOTER }
I20260709 12:22:44.544756 30417 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:44.544948 30417 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:44.545153 30417 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:44.548692 30417 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8175c3d1e5934060aacf4b050f8533bd" member_type: VOTER }
I20260709 12:22:44.549146 30417 leader_election.cc:304] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8175c3d1e5934060aacf4b050f8533bd; no voters: 
I20260709 12:22:44.550145 30417 leader_election.cc:290] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:44.550557 30420 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:44.551779 30420 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 1 LEADER]: Becoming Leader. State: Replica: 8175c3d1e5934060aacf4b050f8533bd, State: Running, Role: LEADER
I20260709 12:22:44.552428 30420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8175c3d1e5934060aacf4b050f8533bd" member_type: VOTER }
I20260709 12:22:44.552987 30417 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:44.558547 30422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8175c3d1e5934060aacf4b050f8533bd. Latest consensus state: current_term: 1 leader_uuid: "8175c3d1e5934060aacf4b050f8533bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8175c3d1e5934060aacf4b050f8533bd" member_type: VOTER } }
I20260709 12:22:44.559080 30422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:44.558923 30421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8175c3d1e5934060aacf4b050f8533bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8175c3d1e5934060aacf4b050f8533bd" member_type: VOTER } }
I20260709 12:22:44.559671 30421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:44.564208 30429 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:44.570083 30429 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:44.571156 29845 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:44.578006 30429 catalog_manager.cc:1383] Generated new cluster ID: e94a442c9f6f46b98888dfc8f85b4009
I20260709 12:22:44.578308 30429 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:44.597494 30429 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:44.598624 30429 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:44.614634 30429 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd: Generated new TSK 0
I20260709 12:22:44.615185 30429 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:44.636956 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:44.642647 30438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.643680 30439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.645051 30441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:44.645464 29845 server_base.cc:1061] running on GCE node
I20260709 12:22:44.646405 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:44.646576 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:44.646698 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599764646687 us; error 0 us; skew 500 ppm
I20260709 12:22:44.647148 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:44.649180 29845 webserver.cc:533] Webserver started at http://127.29.37.65:41355/ using document root <none> and password file <none>
I20260709 12:22:44.649608 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:44.649762 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:44.649978 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:44.651037 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-0-root/instance:
uuid: "55c82dbebacf497290d878424871882b"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-g1m6"
I20260709 12:22:44.655099 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:22:44.658020 30446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.658713 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.002s
I20260709 12:22:44.659024 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "55c82dbebacf497290d878424871882b"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-g1m6"
I20260709 12:22:44.659325 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:44.679847 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:44.680999 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:44.682529 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:44.684655 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:44.684865 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.685097 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:44.685309 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.722885 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:40923
I20260709 12:22:44.722977 30508 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:40923 every 8 connection(s)
I20260709 12:22:44.727802 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:44.735353 30513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.737609 30514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.740439 30516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:44.740836 29845 server_base.cc:1061] running on GCE node
I20260709 12:22:44.741269 30509 heartbeater.cc:344] Connected to a master server at 127.29.37.126:44267
I20260709 12:22:44.741633 30509 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:44.742340 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:22:44.742349 30509 heartbeater.cc:507] Master 127.29.37.126:44267 requested a full tablet report, sending...
W20260709 12:22:44.742663 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:44.742903 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599764742882 us; error 0 us; skew 500 ppm
I20260709 12:22:44.743472 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:44.744787 30382 ts_manager.cc:194] Registered new tserver with Master: 55c82dbebacf497290d878424871882b (127.29.37.65:40923)
I20260709 12:22:44.745921 29845 webserver.cc:533] Webserver started at http://127.29.37.66:44795/ using document root <none> and password file <none>
I20260709 12:22:44.746481 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:44.746690 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:44.746754 30382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39444
I20260709 12:22:44.746987 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:44.748180 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-1-root/instance:
uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-g1m6"
I20260709 12:22:44.752368 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:44.755437 30521 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.756108 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:22:44.756402 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-1-root
uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-g1m6"
I20260709 12:22:44.756683 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:44.782835 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:44.783953 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:44.785300 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:44.787384 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:44.787607 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.787839 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:44.788008 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.825053 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.66:43019
I20260709 12:22:44.825151 30583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.66:43019 every 8 connection(s)
I20260709 12:22:44.829525 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:44.835763 30587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.837443 30588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:44.838938 30590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:44.840111 29845 server_base.cc:1061] running on GCE node
I20260709 12:22:44.840993 30584 heartbeater.cc:344] Connected to a master server at 127.29.37.126:44267
I20260709 12:22:44.841176 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:22:44.841411 30584 heartbeater.cc:461] Registering TS with master...
W20260709 12:22:44.841454 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:44.841837 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599764841818 us; error 0 us; skew 500 ppm
I20260709 12:22:44.842549 30584 heartbeater.cc:507] Master 127.29.37.126:44267 requested a full tablet report, sending...
I20260709 12:22:44.842693 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:44.845053 30382 ts_manager.cc:194] Registered new tserver with Master: 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:44.846107 29845 webserver.cc:533] Webserver started at http://127.29.37.67:40353/ using document root <none> and password file <none>
I20260709 12:22:44.846879 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:44.846894 30382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39458
I20260709 12:22:44.847282 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:44.847689 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:44.849291 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-2-root/instance:
uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-g1m6"
I20260709 12:22:44.853355 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:22:44.856406 30595 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.857076 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:22:44.857307 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-2-root
uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
format_stamp: "Formatted at 2026-07-09 12:22:44 on dist-test-slave-g1m6"
I20260709 12:22:44.857573 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599734091681-29845-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:44.877647 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:44.878786 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:44.880182 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:44.882207 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:44.882524 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:22:44.882769 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:44.882925 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:44.920583 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.67:32939
I20260709 12:22:44.920676 30657 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.67:32939 every 8 connection(s)
I20260709 12:22:44.933318 30658 heartbeater.cc:344] Connected to a master server at 127.29.37.126:44267
I20260709 12:22:44.933653 30658 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:44.934409 30658 heartbeater.cc:507] Master 127.29.37.126:44267 requested a full tablet report, sending...
I20260709 12:22:44.936151 30382 ts_manager.cc:194] Registered new tserver with Master: a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:44.936651 29845 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013116755s
I20260709 12:22:44.937748 30382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39464
I20260709 12:22:44.959955 30382 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39466:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:44.962836 30382 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:45.037678 30549 tablet_service.cc:1511] Processing CreateTablet for tablet 3d5ec309913d4bcba92de3e26da3b30c (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:45.039122 30549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d5ec309913d4bcba92de3e26da3b30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.043617 30548 tablet_service.cc:1511] Processing CreateTablet for tablet 5af4c29d63c344c7be09a1e528d49780 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:45.045367 30548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af4c29d63c344c7be09a1e528d49780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.047726 30547 tablet_service.cc:1511] Processing CreateTablet for tablet 5cce934fbeed4698aa8b79694b17045b (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:45.050736 30547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cce934fbeed4698aa8b79694b17045b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.054428 30546 tablet_service.cc:1511] Processing CreateTablet for tablet 165c70bd1d644a0b8fe47ff2183c841b (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:45.056120 30546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165c70bd1d644a0b8fe47ff2183c841b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.058625 30545 tablet_service.cc:1511] Processing CreateTablet for tablet 5edd6f3bc4164863a96901c816acc154 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:45.060209 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5edd6f3bc4164863a96901c816acc154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.062153 30544 tablet_service.cc:1511] Processing CreateTablet for tablet bfbb1e8ba58b466d9be480f3f2b3a3b7 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:45.063818 30544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfbb1e8ba58b466d9be480f3f2b3a3b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.081504 30623 tablet_service.cc:1511] Processing CreateTablet for tablet 3d5ec309913d4bcba92de3e26da3b30c (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:45.082010 30474 tablet_service.cc:1511] Processing CreateTablet for tablet 3d5ec309913d4bcba92de3e26da3b30c (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:45.083921 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 5af4c29d63c344c7be09a1e528d49780 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:45.085222 30623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d5ec309913d4bcba92de3e26da3b30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.085114 30473 tablet_service.cc:1511] Processing CreateTablet for tablet 5af4c29d63c344c7be09a1e528d49780 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:45.086203 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af4c29d63c344c7be09a1e528d49780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.086613 30474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d5ec309913d4bcba92de3e26da3b30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.094981 30473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af4c29d63c344c7be09a1e528d49780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.113240 30621 tablet_service.cc:1511] Processing CreateTablet for tablet 5cce934fbeed4698aa8b79694b17045b (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:45.113546 30620 tablet_service.cc:1511] Processing CreateTablet for tablet 165c70bd1d644a0b8fe47ff2183c841b (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:45.114656 30621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cce934fbeed4698aa8b79694b17045b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.115798 30472 tablet_service.cc:1511] Processing CreateTablet for tablet 5cce934fbeed4698aa8b79694b17045b (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:45.117069 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cce934fbeed4698aa8b79694b17045b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.117504 30620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165c70bd1d644a0b8fe47ff2183c841b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.124024 30471 tablet_service.cc:1511] Processing CreateTablet for tablet 165c70bd1d644a0b8fe47ff2183c841b (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:45.125272 30471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165c70bd1d644a0b8fe47ff2183c841b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.127763 30619 tablet_service.cc:1511] Processing CreateTablet for tablet 5edd6f3bc4164863a96901c816acc154 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:45.128999 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5edd6f3bc4164863a96901c816acc154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.131753 30470 tablet_service.cc:1511] Processing CreateTablet for tablet 5edd6f3bc4164863a96901c816acc154 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:45.132998 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5edd6f3bc4164863a96901c816acc154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.136335 30673 tablet_bootstrap.cc:492] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.139798 30469 tablet_service.cc:1511] Processing CreateTablet for tablet bfbb1e8ba58b466d9be480f3f2b3a3b7 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:45.141047 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfbb1e8ba58b466d9be480f3f2b3a3b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.142356 30621 tablet_service.cc:1511] Processing CreateTablet for tablet bfbb1e8ba58b466d9be480f3f2b3a3b7 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:45.143411 30673 tablet_bootstrap.cc:654] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.143551 30621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfbb1e8ba58b466d9be480f3f2b3a3b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.151659 30623 tablet_service.cc:1511] Processing CreateTablet for tablet ff1f64f4f9144a8eb7d5a8ca30340c5a (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:45.152832 30623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff1f64f4f9144a8eb7d5a8ca30340c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.153774 30673 tablet_bootstrap.cc:492] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.154296 30673 ts_tablet_manager.cc:1403] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:22:45.159650 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 5c6f64b6e3cd4f688c95b17bd68b4df8 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:45.163619 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c6f64b6e3cd4f688c95b17bd68b4df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.164995 30673 raft_consensus.cc:359] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.165822 30673 raft_consensus.cc:385] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.167856 30543 tablet_service.cc:1511] Processing CreateTablet for tablet ff1f64f4f9144a8eb7d5a8ca30340c5a (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:45.169602 30542 tablet_service.cc:1511] Processing CreateTablet for tablet 5c6f64b6e3cd4f688c95b17bd68b4df8 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:45.170104 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff1f64f4f9144a8eb7d5a8ca30340c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.170909 30542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c6f64b6e3cd4f688c95b17bd68b4df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.175275 30541 tablet_service.cc:1511] Processing CreateTablet for tablet ea907919202a461a824a7edbf698d237 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:45.176527 30541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea907919202a461a824a7edbf698d237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.179366 30540 tablet_service.cc:1511] Processing CreateTablet for tablet 4f6de2e7a11144f3afbcabf3c9cd8802 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:45.180550 30540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f6de2e7a11144f3afbcabf3c9cd8802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.182266 30673 raft_consensus.cc:740] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.183596 30618 tablet_service.cc:1511] Processing CreateTablet for tablet 4f6de2e7a11144f3afbcabf3c9cd8802 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:45.184930 30618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f6de2e7a11144f3afbcabf3c9cd8802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.184613 30673 consensus_queue.cc:260] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.187486 30619 tablet_service.cc:1511] Processing CreateTablet for tablet ea907919202a461a824a7edbf698d237 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:45.188962 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea907919202a461a824a7edbf698d237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.236755 30468 tablet_service.cc:1511] Processing CreateTablet for tablet ea907919202a461a824a7edbf698d237 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:45.238041 30468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea907919202a461a824a7edbf698d237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.237913 30469 tablet_service.cc:1511] Processing CreateTablet for tablet 5c6f64b6e3cd4f688c95b17bd68b4df8 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:45.241290 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c6f64b6e3cd4f688c95b17bd68b4df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.239682 30467 tablet_service.cc:1511] Processing CreateTablet for tablet 4f6de2e7a11144f3afbcabf3c9cd8802 (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:45.242280 30467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f6de2e7a11144f3afbcabf3c9cd8802. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.235747 30471 tablet_service.cc:1511] Processing CreateTablet for tablet ff1f64f4f9144a8eb7d5a8ca30340c5a (DEFAULT_TABLE table=test-workload [id=2e5f3f105cda4205b82933d497f68c1a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:45.255661 30471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff1f64f4f9144a8eb7d5a8ca30340c5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:45.275419 30658 heartbeater.cc:499] Master 127.29.37.126:44267 was elected leader, sending a full tablet report...
I20260709 12:22:45.279625 30673 ts_tablet_manager.cc:1434] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.121s	user 0.026s	sys 0.043s
I20260709 12:22:45.280571 30673 tablet_bootstrap.cc:492] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.291431 30675 tablet_bootstrap.cc:492] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.296890 30677 tablet_bootstrap.cc:492] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.301954 30675 tablet_bootstrap.cc:654] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.293730 30673 tablet_bootstrap.cc:654] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.303027 30677 tablet_bootstrap.cc:654] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.317194 30673 tablet_bootstrap.cc:492] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.317600 30673 ts_tablet_manager.cc:1403] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.004s
I20260709 12:22:45.317734 30675 tablet_bootstrap.cc:492] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.317885 30677 tablet_bootstrap.cc:492] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.318189 30675 ts_tablet_manager.cc:1403] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.005s
I20260709 12:22:45.318310 30677 ts_tablet_manager.cc:1403] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.003s
I20260709 12:22:45.319862 30673 raft_consensus.cc:359] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.320405 30673 raft_consensus.cc:385] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.320641 30673 raft_consensus.cc:740] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.321101 30677 raft_consensus.cc:359] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.321251 30675 raft_consensus.cc:359] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.321322 30673 consensus_queue.cc:260] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.321798 30677 raft_consensus.cc:385] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.321964 30675 raft_consensus.cc:385] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.322173 30677 raft_consensus.cc:740] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.322341 30675 raft_consensus.cc:740] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.322970 30677 consensus_queue.cc:260] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.323236 30675 consensus_queue.cc:260] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.323907 30673 ts_tablet_manager.cc:1434] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.006s
I20260709 12:22:45.325273 30673 tablet_bootstrap.cc:492] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.328722 30509 heartbeater.cc:499] Master 127.29.37.126:44267 was elected leader, sending a full tablet report...
I20260709 12:22:45.329624 30677 ts_tablet_manager.cc:1434] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 12:22:45.330876 30677 tablet_bootstrap.cc:492] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.339411 30677 tablet_bootstrap.cc:654] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.344291 30673 tablet_bootstrap.cc:654] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.354477 30584 heartbeater.cc:499] Master 127.29.37.126:44267 was elected leader, sending a full tablet report...
I20260709 12:22:45.355273 30675 ts_tablet_manager.cc:1434] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.026s
I20260709 12:22:45.356467 30675 tablet_bootstrap.cc:492] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.360152 30673 tablet_bootstrap.cc:492] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.360599 30673 ts_tablet_manager.cc:1403] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.036s	user 0.005s	sys 0.009s
I20260709 12:22:45.363575 30673 raft_consensus.cc:359] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.364446 30675 tablet_bootstrap.cc:654] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.364291 30673 raft_consensus.cc:385] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.365301 30673 raft_consensus.cc:740] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.366094 30673 consensus_queue.cc:260] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.367206 30677 tablet_bootstrap.cc:492] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.367688 30677 ts_tablet_manager.cc:1403] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.014s
I20260709 12:22:45.370536 30677 raft_consensus.cc:359] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.371270 30677 raft_consensus.cc:385] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.371590 30677 raft_consensus.cc:740] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.372072 30673 ts_tablet_manager.cc:1434] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:22:45.372299 30677 consensus_queue.cc:260] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.374732 30677 ts_tablet_manager.cc:1434] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:22:45.375489 30675 tablet_bootstrap.cc:492] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.375542 30676 raft_consensus.cc:493] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.375792 30677 tablet_bootstrap.cc:492] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.376101 30676 raft_consensus.cc:515] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.380020 30676 leader_election.cc:290] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:45.382552 30677 tablet_bootstrap.cc:654] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.373274 30673 tablet_bootstrap.cc:492] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.376084 30675 ts_tablet_manager.cc:1403] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.009s
I20260709 12:22:45.388923 30675 raft_consensus.cc:359] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.390936 30675 raft_consensus.cc:385] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.391618 30675 raft_consensus.cc:740] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.393038 30675 consensus_queue.cc:260] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.393180 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cce934fbeed4698aa8b79694b17045b" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.394361 30681 raft_consensus.cc:493] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:45.395865 30596 leader_election.cc:343] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55c82dbebacf497290d878424871882b (127.29.37.65:40923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.396267 30681 raft_consensus.cc:515] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.398141 30675 ts_tablet_manager.cc:1434] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 12:22:45.399032 30675 tablet_bootstrap.cc:492] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.405025 30675 tablet_bootstrap.cc:654] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.420511 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cce934fbeed4698aa8b79694b17045b" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.422173 30675 tablet_bootstrap.cc:492] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.422570 30677 tablet_bootstrap.cc:492] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.422688 30675 ts_tablet_manager.cc:1403] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.007s
I20260709 12:22:45.422055 30681 leader_election.cc:290] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.423170 30677 ts_tablet_manager.cc:1403] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.048s	user 0.032s	sys 0.012s
W20260709 12:22:45.422468 30599 leader_election.cc:343] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.424065 30599 leader_election.cc:304] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7
I20260709 12:22:45.424885 30676 raft_consensus.cc:2749] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:45.425661 30675 raft_consensus.cc:359] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.425899 30677 raft_consensus.cc:359] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.426240 30673 tablet_bootstrap.cc:654] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.426441 30675 raft_consensus.cc:385] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.426596 30677 raft_consensus.cc:385] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.426853 30675 raft_consensus.cc:740] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.426981 30677 raft_consensus.cc:740] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.427659 30675 consensus_queue.cc:260] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.427728 30677 consensus_queue.cc:260] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.429917 30675 ts_tablet_manager.cc:1434] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:45.430886 30675 tablet_bootstrap.cc:492] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.433995 30677 ts_tablet_manager.cc:1434] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:45.437218 30675 tablet_bootstrap.cc:654] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.437538 30677 tablet_bootstrap.cc:492] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.452342 30677 tablet_bootstrap.cc:654] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.455601 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.456197 30559 raft_consensus.cc:2468] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 0.
I20260709 12:22:45.456506 30675 tablet_bootstrap.cc:492] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.457008 30675 ts_tablet_manager.cc:1403] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.012s
I20260709 12:22:45.458907 30450 leader_election.cc:304] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:45.459739 30681 raft_consensus.cc:2804] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.460209 30681 raft_consensus.cc:493] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.460026 30675 raft_consensus.cc:359] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.460736 30681 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.460928 30675 raft_consensus.cc:385] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.461362 30675 raft_consensus.cc:740] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.462137 30675 consensus_queue.cc:260] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.467008 30675 ts_tablet_manager.cc:1434] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:45.468003 30675 tablet_bootstrap.cc:492] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.471199 30681 raft_consensus.cc:515] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.474427 30695 raft_consensus.cc:493] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.474881 30695 raft_consensus.cc:515] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.475987 30675 tablet_bootstrap.cc:654] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.473316 30673 tablet_bootstrap.cc:492] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.477094 30673 ts_tablet_manager.cc:1403] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.104s	user 0.020s	sys 0.015s
I20260709 12:22:45.477730 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.478631 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
W20260709 12:22:45.479105 30450 leader_election.cc:343] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.479525 30695 leader_election.cc:290] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.479853 30677 tablet_bootstrap.cc:492] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.480010 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.480289 30677 ts_tablet_manager.cc:1403] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.043s	user 0.007s	sys 0.014s
W20260709 12:22:45.481168 30448 leader_election.cc:343] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.481781 30681 leader_election.cc:290] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.482455 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:45.482925 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:45.482996 30559 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.483186 30677 raft_consensus.cc:359] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.483939 30677 raft_consensus.cc:385] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:22:45.484290 30448 leader_election.cc:343] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.484344 30673 raft_consensus.cc:359] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.484938 30448 leader_election.cc:304] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55c82dbebacf497290d878424871882b; no voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:22:45.485296 30673 raft_consensus.cc:385] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.485630 30673 raft_consensus.cc:740] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.484236 30677 raft_consensus.cc:740] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
W20260709 12:22:45.486100 30448 leader_election.cc:343] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.486377 30673 consensus_queue.cc:260] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.486853 30681 raft_consensus.cc:2749] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:45.486794 30677 consensus_queue.cc:260] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.489048 30677 ts_tablet_manager.cc:1434] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:22:45.489925 30677 tablet_bootstrap.cc:492] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.491025 30559 raft_consensus.cc:2468] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 1.
I20260709 12:22:45.492173 30450 leader_election.cc:304] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:22:45.493259 30681 raft_consensus.cc:2804] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.494062 30681 raft_consensus.cc:697] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:45.495025 30673 ts_tablet_manager.cc:1434] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:22:45.494964 30681 consensus_queue.cc:237] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.496022 30677 tablet_bootstrap.cc:654] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.502393 30673 tablet_bootstrap.cc:492] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.502590 30675 tablet_bootstrap.cc:492] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.503012 30675 ts_tablet_manager.cc:1403] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.000s
I20260709 12:22:45.503646 30677 tablet_bootstrap.cc:492] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.504125 30677 ts_tablet_manager.cc:1403] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 12:22:45.505831 30675 raft_consensus.cc:359] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.506425 30676 raft_consensus.cc:493] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.506629 30675 raft_consensus.cc:385] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.506948 30675 raft_consensus.cc:740] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.506902 30676 raft_consensus.cc:515] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.506986 30677 raft_consensus.cc:359] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.507934 30677 raft_consensus.cc:385] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.508311 30677 raft_consensus.cc:740] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.507953 30675 consensus_queue.cc:260] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.509049 30676 leader_election.cc:290] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:45.510478 30675 ts_tablet_manager.cc:1434] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:45.510582 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.511260 30559 raft_consensus.cc:2468] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 0.
I20260709 12:22:45.511514 30675 tablet_bootstrap.cc:492] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.510586 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.512248 30599 leader_election.cc:304] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.513142 30676 raft_consensus.cc:2804] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:22:45.513307 30596 leader_election.cc:343] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55c82dbebacf497290d878424871882b (127.29.37.65:40923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.513507 30676 raft_consensus.cc:493] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.513875 30676 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.514381 30677 consensus_queue.cc:260] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.517050 30677 ts_tablet_manager.cc:1434] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.004s
I20260709 12:22:45.517947 30677 tablet_bootstrap.cc:492] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.518519 30675 tablet_bootstrap.cc:654] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.521911 30695 raft_consensus.cc:493] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.526523 30695 raft_consensus.cc:515] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.527256 30699 raft_consensus.cc:493] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.527805 30699 raft_consensus.cc:515] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.528689 30695 leader_election.cc:290] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.529337 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cce934fbeed4698aa8b79694b17045b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.529924 30559 raft_consensus.cc:2468] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 0.
I20260709 12:22:45.530294 30675 tablet_bootstrap.cc:492] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.530841 30675 ts_tablet_manager.cc:1403] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.000s
I20260709 12:22:45.523857 30677 tablet_bootstrap.cc:654] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.523986 30673 tablet_bootstrap.cc:654] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.531071 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cce934fbeed4698aa8b79694b17045b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.531687 30632 raft_consensus.cc:2468] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 0.
I20260709 12:22:45.521713 30676 raft_consensus.cc:515] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.532958 30448 leader_election.cc:304] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.530642 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.533819 30558 raft_consensus.cc:2468] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 0.
I20260709 12:22:45.534199 30676 leader_election.cc:290] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:45.535672 30695 raft_consensus.cc:2804] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.535831 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:45.536042 30695 raft_consensus.cc:493] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.536388 30695 raft_consensus.cc:3060] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.536409 30599 leader_election.cc:304] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
W20260709 12:22:45.537276 30596 leader_election.cc:343] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55c82dbebacf497290d878424871882b (127.29.37.65:40923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.537776 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:45.538348 30558 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.540143 30675 raft_consensus.cc:359] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.540867 30675 raft_consensus.cc:385] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.541170 30675 raft_consensus.cc:740] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.540562 30382 catalog_manager.cc:5697] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 0 to 1, leader changed from <none> to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 1 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.541914 30675 consensus_queue.cc:260] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.543725 30695 raft_consensus.cc:515] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.546427 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cce934fbeed4698aa8b79694b17045b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:45.546947 30559 raft_consensus.cc:3060] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.549417 30676 raft_consensus.cc:2804] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.549777 30676 raft_consensus.cc:493] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.550187 30675 ts_tablet_manager.cc:1434] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:22:45.550103 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.551249 30675 tablet_bootstrap.cc:492] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
W20260709 12:22:45.551913 30596 leader_election.cc:343] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55c82dbebacf497290d878424871882b (127.29.37.65:40923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.552131 30699 leader_election.cc:290] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), 55c82dbebacf497290d878424871882b (127.29.37.65:40923)
I20260709 12:22:45.553251 30559 raft_consensus.cc:2468] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 1.
I20260709 12:22:45.551908 30558 raft_consensus.cc:2468] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 1.
I20260709 12:22:45.554419 30450 leader_election.cc:304] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:45.555639 30681 raft_consensus.cc:2804] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.557097 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cce934fbeed4698aa8b79694b17045b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:45.557662 30632 raft_consensus.cc:3060] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.550127 30676 raft_consensus.cc:3060] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.564435 30675 tablet_bootstrap.cc:654] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.569015 30681 raft_consensus.cc:697] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:45.568888 30676 raft_consensus.cc:515] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.570178 30677 tablet_bootstrap.cc:492] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.569949 30681 consensus_queue.cc:237] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.570724 30677 ts_tablet_manager.cc:1403] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.053s	user 0.020s	sys 0.009s
I20260709 12:22:45.571959 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b"
W20260709 12:22:45.573287 30596 leader_election.cc:343] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55c82dbebacf497290d878424871882b (127.29.37.65:40923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:45.573858 30676 leader_election.cc:290] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), 55c82dbebacf497290d878424871882b (127.29.37.65:40923)
I20260709 12:22:45.573619 30677 raft_consensus.cc:359] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.574600 30677 raft_consensus.cc:385] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.574987 30677 raft_consensus.cc:740] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.564323 30632 raft_consensus.cc:2468] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 1.
I20260709 12:22:45.564891 30695 leader_election.cc:290] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
W20260709 12:22:45.565750 30430 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:45.567361 30673 tablet_bootstrap.cc:492] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.575944 30677 consensus_queue.cc:260] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.576923 30673 ts_tablet_manager.cc:1403] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.078s	user 0.015s	sys 0.003s
I20260709 12:22:45.578536 30677 ts_tablet_manager.cc:1434] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:22:45.579444 30677 tablet_bootstrap.cc:492] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.567417 30599 leader_election.cc:304] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 55c82dbebacf497290d878424871882b
I20260709 12:22:45.582429 30676 raft_consensus.cc:2804] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.583016 30676 raft_consensus.cc:697] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:45.583701 30673 raft_consensus.cc:359] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.583988 30676 consensus_queue.cc:237] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.585263 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:45.585803 30558 raft_consensus.cc:3060] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.585991 30677 tablet_bootstrap.cc:654] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.584654 30673 raft_consensus.cc:385] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.590399 30673 raft_consensus.cc:740] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.591346 30673 consensus_queue.cc:260] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.593042 30675 tablet_bootstrap.cc:492] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.593495 30675 ts_tablet_manager.cc:1403] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.007s
I20260709 12:22:45.593940 30673 ts_tablet_manager.cc:1434] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.005s
I20260709 12:22:45.594015 30558 raft_consensus.cc:2468] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 1.
I20260709 12:22:45.595634 30599 leader_election.cc:304] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 55c82dbebacf497290d878424871882b
I20260709 12:22:45.596293 30675 raft_consensus.cc:359] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.596724 30699 raft_consensus.cc:2804] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.596980 30675 raft_consensus.cc:385] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.597313 30675 raft_consensus.cc:740] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.598379 30673 tablet_bootstrap.cc:492] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.603478 30677 tablet_bootstrap.cc:492] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.604065 30677 ts_tablet_manager.cc:1403] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.001s
I20260709 12:22:45.607407 30677 raft_consensus.cc:359] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.608124 30677 raft_consensus.cc:385] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.608621 30677 raft_consensus.cc:740] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.608129 30382 catalog_manager.cc:5697] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 0 to 1, leader changed from <none> to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 1 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.609548 30677 consensus_queue.cc:260] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.611817 30673 tablet_bootstrap.cc:654] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.612031 30677 ts_tablet_manager.cc:1434] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:22:45.617206 30677 tablet_bootstrap.cc:492] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.613018 30675 consensus_queue.cc:260] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.615242 30699 raft_consensus.cc:697] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:45.622632 30677 tablet_bootstrap.cc:654] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.622323 30380 catalog_manager.cc:5697] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 0 to 1, leader changed from <none> to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 1 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:45.623030 30699 consensus_queue.cc:237] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.632042 30675 ts_tablet_manager.cc:1434] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:22:45.632697 30681 raft_consensus.cc:493] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.633059 30675 tablet_bootstrap.cc:492] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.633144 30681 raft_consensus.cc:515] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:45.635505 30681 leader_election.cc:290] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.637055 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.637255 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.638022 30558 raft_consensus.cc:2393] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55c82dbebacf497290d878424871882b in current term 1: Already voted for candidate a504e6da07bd4feb8ec83d2b4ae4706c in this term.
I20260709 12:22:45.639436 30450 leader_election.cc:304] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55c82dbebacf497290d878424871882b; no voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:22:45.640784 30681 raft_consensus.cc:3060] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.644634 30675 tablet_bootstrap.cc:654] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.649112 30681 raft_consensus.cc:2749] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:45.650292 30677 tablet_bootstrap.cc:492] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.651274 30677 ts_tablet_manager.cc:1403] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.012s
I20260709 12:22:45.653668 30675 tablet_bootstrap.cc:492] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.650331 30673 tablet_bootstrap.cc:492] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.654142 30675 ts_tablet_manager.cc:1403] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.000s
I20260709 12:22:45.654737 30673 ts_tablet_manager.cc:1403] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.008s
I20260709 12:22:45.654179 30380 catalog_manager.cc:5697] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 0 to 1, leader changed from <none> to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 1 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.657027 30675 raft_consensus.cc:359] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.657722 30675 raft_consensus.cc:385] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.658030 30675 raft_consensus.cc:740] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.653960 30677 raft_consensus.cc:359] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.658612 30677 raft_consensus.cc:385] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.658915 30677 raft_consensus.cc:740] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.659021 30675 consensus_queue.cc:260] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.659565 30677 consensus_queue.cc:260] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.660390 30673 raft_consensus.cc:359] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.661059 30673 raft_consensus.cc:385] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.661360 30673 raft_consensus.cc:740] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.662048 30673 consensus_queue.cc:260] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.663734 30675 ts_tablet_manager.cc:1434] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:22:45.664161 30673 ts_tablet_manager.cc:1434] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.004s
I20260709 12:22:45.664703 30675 tablet_bootstrap.cc:492] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.665759 30673 tablet_bootstrap.cc:492] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.666415 30677 ts_tablet_manager.cc:1434] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:22:45.667307 30677 tablet_bootstrap.cc:492] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.672772 30677 tablet_bootstrap.cc:654] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.673202 30673 tablet_bootstrap.cc:654] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.676234 30675 tablet_bootstrap.cc:654] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.678139 30699 raft_consensus.cc:493] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.678646 30699 raft_consensus.cc:515] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.681298 30699 leader_election.cc:290] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:45.682418 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.683051 30558 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 0.
I20260709 12:22:45.684242 30599 leader_election.cc:304] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.685392 30699 raft_consensus.cc:2804] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.685895 30699 raft_consensus.cc:493] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.686374 30699 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.686324 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.687245 30484 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 0.
I20260709 12:22:45.700515 30711 raft_consensus.cc:493] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.700484 30699 raft_consensus.cc:515] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.701186 30673 tablet_bootstrap.cc:492] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.701664 30677 tablet_bootstrap.cc:492] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.701174 30711 raft_consensus.cc:515] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.701790 30673 ts_tablet_manager.cc:1403] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.013s
I20260709 12:22:45.702113 30677 ts_tablet_manager.cc:1403] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.015s
I20260709 12:22:45.703383 30711 leader_election.cc:290] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:45.703737 30699 leader_election.cc:290] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:45.704797 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.704828 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:45.704999 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.704674 30677 raft_consensus.cc:359] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.703279 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:45.705555 30558 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.705714 30677 raft_consensus.cc:385] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.705494 30559 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 0.
I20260709 12:22:45.705648 30483 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 0.
I20260709 12:22:45.706177 30677 raft_consensus.cc:740] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.706045 30484 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.706887 30677 consensus_queue.cc:260] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.707199 30596 leader_election.cc:304] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.708046 30699 raft_consensus.cc:2804] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.708379 30699 raft_consensus.cc:493] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.708698 30699 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.709501 30675 tablet_bootstrap.cc:492] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.709941 30675 ts_tablet_manager.cc:1403] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.006s
I20260709 12:22:45.712411 30675 raft_consensus.cc:359] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.712992 30675 raft_consensus.cc:385] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.713261 30675 raft_consensus.cc:740] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.713629 30484 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 1.
I20260709 12:22:45.714031 30675 consensus_queue.cc:260] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.713164 30558 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 1.
I20260709 12:22:45.714900 30699 raft_consensus.cc:515] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.715032 30596 leader_election.cc:304] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.716076 30675 ts_tablet_manager.cc:1434] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:45.716212 30711 raft_consensus.cc:2804] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.716616 30711 raft_consensus.cc:697] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:45.716928 30675 tablet_bootstrap.cc:492] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7: Bootstrap starting.
I20260709 12:22:45.716737 30673 raft_consensus.cc:359] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.717486 30673 raft_consensus.cc:385] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.717382 30711 consensus_queue.cc:237] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.717803 30673 raft_consensus.cc:740] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.719619 30677 ts_tablet_manager.cc:1434] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:22:45.719497 30673 consensus_queue.cc:260] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.720507 30677 tablet_bootstrap.cc:492] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b: Bootstrap starting.
I20260709 12:22:45.721807 30673 ts_tablet_manager.cc:1434] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260709 12:22:45.722759 30673 tablet_bootstrap.cc:492] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.724598 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:45.725150 30484 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.725661 30699 leader_election.cc:290] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:45.726367 30677 tablet_bootstrap.cc:654] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.730619 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:45.731226 30558 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.735743 30675 tablet_bootstrap.cc:654] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.737196 30558 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 1.
I20260709 12:22:45.738075 30599 leader_election.cc:304] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.738901 30699 raft_consensus.cc:2804] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.739415 30484 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 1.
I20260709 12:22:45.739816 30699 raft_consensus.cc:697] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:45.740931 30699 consensus_queue.cc:237] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.747184 30673 tablet_bootstrap.cc:654] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.749665 30677 tablet_bootstrap.cc:492] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b: No bootstrap required, opened a new log
I20260709 12:22:45.750160 30677 ts_tablet_manager.cc:1403] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.004s
I20260709 12:22:45.747416 30675 tablet_bootstrap.cc:492] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7: No bootstrap required, opened a new log
I20260709 12:22:45.751916 30675 ts_tablet_manager.cc:1403] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.000s
I20260709 12:22:45.753026 30677 raft_consensus.cc:359] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.752558 30382 catalog_manager.cc:5697] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 0 to 1, leader changed from <none> to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 1 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:45.753770 30677 raft_consensus.cc:385] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.754108 30677 raft_consensus.cc:740] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.754511 30675 raft_consensus.cc:359] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.755250 30673 tablet_bootstrap.cc:492] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.755795 30673 ts_tablet_manager.cc:1403] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.000s
I20260709 12:22:45.755002 30677 consensus_queue.cc:260] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.755239 30675 raft_consensus.cc:385] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.758148 30675 raft_consensus.cc:740] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.758194 30673 raft_consensus.cc:359] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.758981 30673 raft_consensus.cc:385] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.759306 30673 raft_consensus.cc:740] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.759148 30675 consensus_queue.cc:260] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.760017 30673 consensus_queue.cc:260] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.760494 30677 ts_tablet_manager.cc:1434] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:22:45.762187 30673 ts_tablet_manager.cc:1434] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 12:22:45.763087 30673 tablet_bootstrap.cc:492] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.763408 30675 ts_tablet_manager.cc:1434] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:22:45.768523 30673 tablet_bootstrap.cc:654] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.773561 30673 tablet_bootstrap.cc:492] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.773950 30673 ts_tablet_manager.cc:1403] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:22:45.774253 30380 catalog_manager.cc:5697] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 0 to 1, leader changed from <none> to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 1 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:45.776196 30673 raft_consensus.cc:359] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.776633 30673 raft_consensus.cc:385] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.776855 30673 raft_consensus.cc:740] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.777383 30673 consensus_queue.cc:260] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.778739 30673 ts_tablet_manager.cc:1434] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.003s
I20260709 12:22:45.778856 30681 raft_consensus.cc:493] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.779559 30673 tablet_bootstrap.cc:492] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c: Bootstrap starting.
I20260709 12:22:45.779268 30681 raft_consensus.cc:515] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.781235 30681 leader_election.cc:290] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.782275 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.782697 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.782831 30632 raft_consensus.cc:2468] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 0.
I20260709 12:22:45.783262 30558 raft_consensus.cc:2468] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 0.
I20260709 12:22:45.784145 30448 leader_election.cc:304] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.784782 30681 raft_consensus.cc:2804] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.785060 30673 tablet_bootstrap.cc:654] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:45.785221 30681 raft_consensus.cc:493] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.785601 30681 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.791738 30681 raft_consensus.cc:515] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.791927 30673 tablet_bootstrap.cc:492] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c: No bootstrap required, opened a new log
I20260709 12:22:45.792477 30673 ts_tablet_manager.cc:1403] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 12:22:45.793402 30681 leader_election.cc:290] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.794133 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:45.794402 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:45.794677 30558 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.794924 30632 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.794831 30673 raft_consensus.cc:359] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.795562 30673 raft_consensus.cc:385] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:45.795826 30673 raft_consensus.cc:740] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Initialized, Role: FOLLOWER
I20260709 12:22:45.796468 30673 consensus_queue.cc:260] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.798456 30673 ts_tablet_manager.cc:1434] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:22:45.800608 30632 raft_consensus.cc:2468] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 1.
I20260709 12:22:45.800736 30558 raft_consensus.cc:2468] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 1.
I20260709 12:22:45.801504 30448 leader_election.cc:304] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.802241 30681 raft_consensus.cc:2804] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.802675 30681 raft_consensus.cc:697] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:45.803352 30681 consensus_queue.cc:237] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.809268 30682 raft_consensus.cc:493] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.809713 30682 raft_consensus.cc:515] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.811949 30682 leader_election.cc:290] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.811826 30381 catalog_manager.cc:5697] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 0 to 1, leader changed from <none> to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 1 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.816831 30681 raft_consensus.cc:493] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.817359 30681 raft_consensus.cc:515] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.821260 30681 leader_election.cc:290] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.821789 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.821892 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.822564 30558 raft_consensus.cc:2393] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55c82dbebacf497290d878424871882b in current term 1: Already voted for candidate a504e6da07bd4feb8ec83d2b4ae4706c in this term.
I20260709 12:22:45.824023 30450 leader_election.cc:304] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55c82dbebacf497290d878424871882b; no voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:22:45.824980 30681 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.828825 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.829455 30484 raft_consensus.cc:2468] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 0.
I20260709 12:22:45.830602 30522 leader_election.cc:304] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:45.831034 30681 raft_consensus.cc:2749] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:45.831259 30682 raft_consensus.cc:2804] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.831645 30682 raft_consensus.cc:493] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.831943 30682 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.835430 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.835917 30632 raft_consensus.cc:2468] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 0.
I20260709 12:22:45.837755 30682 raft_consensus.cc:515] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.839749 30681 raft_consensus.cc:493] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.839833 30729 raft_consensus.cc:493] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.840348 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:45.840718 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:45.840339 30681 raft_consensus.cc:515] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.840396 30729 raft_consensus.cc:515] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.841207 30484 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.841296 30632 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.842675 30681 leader_election.cc:290] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.843640 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.844209 30633 raft_consensus.cc:2468] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 0.
I20260709 12:22:45.844790 30729 leader_election.cc:290] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.845155 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.845386 30448 leader_election.cc:304] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.845793 30631 raft_consensus.cc:2468] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 0.
I20260709 12:22:45.845899 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.846195 30681 raft_consensus.cc:2804] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.844703 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.846560 30681 raft_consensus.cc:493] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.846817 30558 raft_consensus.cc:2468] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 0.
I20260709 12:22:45.846947 30681 raft_consensus.cc:3060] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.847186 30523 leader_election.cc:304] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:45.848109 30729 raft_consensus.cc:2804] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.848489 30729 raft_consensus.cc:493] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.848855 30729 raft_consensus.cc:3060] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.850750 30484 raft_consensus.cc:2468] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 1.
I20260709 12:22:45.850827 30682 leader_election.cc:290] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.850750 30632 raft_consensus.cc:2468] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 1.
I20260709 12:22:45.852206 30522 leader_election.cc:304] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:45.853343 30682 raft_consensus.cc:2804] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.853741 30681 raft_consensus.cc:515] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.854537 30682 raft_consensus.cc:697] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:22:45.854952 30483 raft_consensus.cc:2393] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in current term 1: Already voted for candidate 55c82dbebacf497290d878424871882b in this term.
I20260709 12:22:45.855471 30729 raft_consensus.cc:515] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.855507 30682 consensus_queue.cc:237] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.857441 30681 leader_election.cc:290] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.857824 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:45.858351 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:45.858579 30558 raft_consensus.cc:2393] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 55c82dbebacf497290d878424871882b in current term 1: Already voted for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in this term.
I20260709 12:22:45.859001 30483 raft_consensus.cc:2393] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in current term 1: Already voted for candidate 55c82dbebacf497290d878424871882b in this term.
I20260709 12:22:45.859603 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:45.860023 30632 raft_consensus.cc:3060] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.861279 30729 leader_election.cc:290] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.864976 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:45.865155 30632 raft_consensus.cc:2468] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 1.
I20260709 12:22:45.864897 30381 catalog_manager.cc:5697] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.866384 30448 leader_election.cc:304] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 58f1cc3ad9144408a7ade4db8ceffcc7
I20260709 12:22:45.866869 30523 leader_election.cc:304] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:22:45.867199 30681 raft_consensus.cc:2804] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.867707 30681 raft_consensus.cc:697] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:45.868018 30730 raft_consensus.cc:2749] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:22:45.868453 30681 consensus_queue.cc:237] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.876183 30380 catalog_manager.cc:5697] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 0 to 1, leader changed from <none> to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 1 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.893908 30730 raft_consensus.cc:493] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.894352 30730 raft_consensus.cc:515] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.895985 30730 leader_election.cc:290] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.896646 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea907919202a461a824a7edbf698d237" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.896934 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea907919202a461a824a7edbf698d237" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" is_pre_election: true
I20260709 12:22:45.897168 30483 raft_consensus.cc:2468] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 0.
I20260709 12:22:45.897464 30632 raft_consensus.cc:2468] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 0.
I20260709 12:22:45.898136 30522 leader_election.cc:304] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:45.898917 30730 raft_consensus.cc:2804] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:45.899156 30730 raft_consensus.cc:493] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.899379 30730 raft_consensus.cc:3060] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.903640 30730 raft_consensus.cc:515] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.903836 30699 raft_consensus.cc:493] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.904305 30699 raft_consensus.cc:515] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.905157 30730 leader_election.cc:290] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:45.905897 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea907919202a461a824a7edbf698d237" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:45.906101 30699 leader_election.cc:290] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:45.906181 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea907919202a461a824a7edbf698d237" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:45.906451 30483 raft_consensus.cc:3060] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.906805 30632 raft_consensus.cc:3060] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.906771 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c82dbebacf497290d878424871882b" is_pre_election: true
I20260709 12:22:45.907742 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" is_pre_election: true
I20260709 12:22:45.908488 30558 raft_consensus.cc:2393] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a504e6da07bd4feb8ec83d2b4ae4706c in current term 1: Already voted for candidate 55c82dbebacf497290d878424871882b in this term.
I20260709 12:22:45.909415 30599 leader_election.cc:304] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7
I20260709 12:22:45.910172 30699 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:45.912210 30483 raft_consensus.cc:2468] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 1.
I20260709 12:22:45.913067 30632 raft_consensus.cc:2468] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 1.
I20260709 12:22:45.913200 30522 leader_election.cc:304] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:45.913990 30730 raft_consensus.cc:2804] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:45.914538 30730 raft_consensus.cc:697] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:22:45.915377 30730 consensus_queue.cc:237] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:45.917258 30699 raft_consensus.cc:2749] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:45.923666 30380 catalog_manager.cc:5697] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:45.932356 30681 consensus_queue.cc:1048] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:45.941972 30695 consensus_queue.cc:1048] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:45.970088 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:44406
I20260709 12:22:45.970799 30632 raft_consensus.cc:606] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:45.975961 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:44406
I20260709 12:22:45.976485 30632 raft_consensus.cc:606] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:45.977867 30699 consensus_queue.cc:1048] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:45.984685 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:44406
I20260709 12:22:45.985214 30632 raft_consensus.cc:606] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:45.986394 30711 raft_consensus.cc:993] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:45.986811 30711 raft_consensus.cc:1081] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:45.990641 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:44406
I20260709 12:22:45.991173 30632 raft_consensus.cc:606] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:45.990929 30699 consensus_queue.cc:1048] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:45.992352 30484 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49360
I20260709 12:22:45.992934 30484 raft_consensus.cc:493] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:45.993337 30484 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.001771 30484 raft_consensus.cc:515] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.003134 30483 raft_consensus.cc:1240] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Rejecting Update request from peer a504e6da07bd4feb8ec83d2b4ae4706c for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:46.007005 30482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
 from {username='slave'} at 127.0.0.1:49390
I20260709 12:22:46.007567 30482 raft_consensus.cc:606] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Received request to transfer leadership to TS 58f1cc3ad9144408a7ade4db8ceffcc7
I20260709 12:22:46.006880 30711 consensus_queue.cc:1059] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.008692 30711 raft_consensus.cc:3055] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.009052 30711 raft_consensus.cc:740] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:46.009593 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:46.010236 30558 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.009850 30711 consensus_queue.cc:260] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.008693 30484 leader_election.cc:290] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:46.011004 30711 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.014123 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:46.018105 30558 raft_consensus.cc:2468] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:46.019366 30450 leader_election.cc:304] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:46.020198 30681 raft_consensus.cc:2804] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.018468 30484 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
 from {username='slave'} at 127.0.0.1:49390
I20260709 12:22:46.020673 30681 raft_consensus.cc:697] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:46.021059 30484 raft_consensus.cc:606] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Received request to transfer leadership to TS 58f1cc3ad9144408a7ade4db8ceffcc7
I20260709 12:22:46.021497 30681 consensus_queue.cc:237] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.023267 30632 raft_consensus.cc:2468] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:46.027500 30695 consensus_queue.cc:1048] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.029865 30381 catalog_manager.cc:5697] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 1 to 2, leader changed from a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 2 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.036267 30484 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:49390
I20260709 12:22:46.036723 30484 raft_consensus.cc:606] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Received request to transfer leadership to TS a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:22:46.046332 30695 consensus_queue.cc:1048] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.125010 30711 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.135067 30711 consensus_queue.cc:1048] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.135445 30699 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.152680 30699 raft_consensus.cc:993] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:46.153103 30699 raft_consensus.cc:1081] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:46.155560 30484 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49360
I20260709 12:22:46.156291 30484 raft_consensus.cc:493] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.156770 30484 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.176178 30748 consensus_queue.cc:1048] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:46.179420 30484 raft_consensus.cc:515] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.182286 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:46.182819 30558 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.184031 30748 raft_consensus.cc:993] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:46.184398 30484 leader_election.cc:290] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:46.185083 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:46.185585 30699 raft_consensus.cc:1081] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:46.185778 30632 raft_consensus.cc:3055] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.186126 30632 raft_consensus.cc:740] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:46.186950 30632 consensus_queue.cc:260] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.187335 30482 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49360
I20260709 12:22:46.187836 30482 raft_consensus.cc:493] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.188151 30632 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.188170 30482 raft_consensus.cc:3060] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.189888 30558 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:46.190848 30450 leader_election.cc:304] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:46.191500 30695 raft_consensus.cc:2804] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.191908 30695 raft_consensus.cc:697] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:46.192587 30695 consensus_queue.cc:237] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.194885 30632 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:46.194916 30482 raft_consensus.cc:515] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:46.196885 30482 leader_election.cc:290] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:46.197649 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:46.198185 30558 raft_consensus.cc:3060] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.198388 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d5ec309913d4bcba92de3e26da3b30c" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:46.198964 30632 raft_consensus.cc:3055] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.199285 30632 raft_consensus.cc:740] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:46.200085 30632 consensus_queue.cc:260] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:46.201321 30632 raft_consensus.cc:3060] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.203320 30558 raft_consensus.cc:2468] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:46.204201 30450 leader_election.cc:304] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:46.204917 30695 raft_consensus.cc:2804] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.205343 30695 raft_consensus.cc:697] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:46.206107 30695 consensus_queue.cc:237] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } }
I20260709 12:22:46.207055 30632 raft_consensus.cc:2468] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:46.207161 30379 catalog_manager.cc:5697] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 1 to 2, leader changed from a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 2 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.217689 30748 consensus_queue.cc:1048] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.219405 30379 catalog_manager.cc:5697] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 1 to 2, leader changed from a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 2 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:46.224072 30676 raft_consensus.cc:993] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:46.224560 30711 raft_consensus.cc:1081] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:46.226773 30484 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49360
I20260709 12:22:46.227902 30484 raft_consensus.cc:493] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.228332 30484 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.229405 30711 consensus_queue.cc:1048] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.234733 30703 consensus_queue.cc:1048] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:46.235972 30484 raft_consensus.cc:515] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.238442 30484 leader_election.cc:290] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:46.238967 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:46.239537 30632 raft_consensus.cc:3055] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.239880 30632 raft_consensus.cc:740] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:46.240756 30632 consensus_queue.cc:260] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.242573 30632 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.244858 30681 raft_consensus.cc:993] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b: : Instructing follower 58f1cc3ad9144408a7ade4db8ceffcc7 to start an election
I20260709 12:22:46.245296 30681 raft_consensus.cc:1081] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Signalling peer 58f1cc3ad9144408a7ade4db8ceffcc7 to start an election
I20260709 12:22:46.246381 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:46.246917 30558 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.250500 30695 consensus_queue.cc:1048] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.253485 30559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8"
dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
 from {username='slave'} at 127.0.0.1:56446
I20260709 12:22:46.253659 30558 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:46.254076 30559 raft_consensus.cc:493] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.254608 30559 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.255098 30450 leader_election.cc:304] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:46.259087 30703 raft_consensus.cc:2804] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.259539 30703 raft_consensus.cc:697] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:46.260890 30632 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:46.260349 30703 consensus_queue.cc:237] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.264816 30559 raft_consensus.cc:515] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.268010 30559 leader_election.cc:290] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 2 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:46.269222 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:46.269781 30632 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.269096 30379 catalog_manager.cc:5697] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 1 to 2, leader changed from a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 2 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.273344 30703 consensus_queue.cc:1048] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.273608 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:46.274173 30484 raft_consensus.cc:3055] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.274511 30484 raft_consensus.cc:740] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:46.275417 30484 consensus_queue.cc:260] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.276729 30484 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.281324 30730 consensus_queue.cc:1048] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.283247 30484 raft_consensus.cc:2468] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 2.
I20260709 12:22:46.284164 30632 raft_consensus.cc:2468] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 2.
I20260709 12:22:46.284252 30522 leader_election.cc:304] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:46.286346 30730 raft_consensus.cc:2804] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.286907 30730 raft_consensus.cc:697] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Becoming Leader. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:22:46.287950 30730 consensus_queue.cc:237] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.297375 30681 consensus_queue.cc:1048] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.300561 30730 consensus_queue.cc:1048] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.302516 30379 catalog_manager.cc:5697] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 reported cstate change: term changed from 1 to 2, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.316005 30681 raft_consensus.cc:993] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b: : Instructing follower a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:22:46.316425 30681 raft_consensus.cc:1081] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Signalling peer a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:22:46.317027 30729 consensus_queue.cc:1048] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.318826 30632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:44396
I20260709 12:22:46.319350 30632 raft_consensus.cc:493] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.319734 30632 raft_consensus.cc:3060] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.326591 30632 raft_consensus.cc:515] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.329023 30632 leader_election.cc:290] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 2 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:46.330076 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:46.330410 30682 consensus_queue.cc:1048] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:46.330698 30559 raft_consensus.cc:3060] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.336150 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6de2e7a11144f3afbcabf3c9cd8802" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:46.336685 30482 raft_consensus.cc:3055] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.337000 30482 raft_consensus.cc:740] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:46.338063 30482 consensus_queue.cc:260] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.339854 30482 raft_consensus.cc:3060] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.337810 30559 raft_consensus.cc:2468] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 2.
I20260709 12:22:46.347204 30599 leader_election.cc:304] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:46.347358 30482 raft_consensus.cc:2468] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 2.
I20260709 12:22:46.348067 30711 raft_consensus.cc:2804] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.348569 30711 raft_consensus.cc:697] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:46.349270 30711 consensus_queue.cc:237] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.359490 30695 raft_consensus.cc:993] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b: : Instructing follower 58f1cc3ad9144408a7ade4db8ceffcc7 to start an election
I20260709 12:22:46.359918 30681 raft_consensus.cc:1081] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Signalling peer 58f1cc3ad9144408a7ade4db8ceffcc7 to start an election
I20260709 12:22:46.361344 30559 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7"
dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
 from {username='slave'} at 127.0.0.1:56446
I20260709 12:22:46.361876 30559 raft_consensus.cc:493] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:46.362210 30559 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.364966 30379 catalog_manager.cc:5697] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 1 to 2, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 2 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:46.368564 30559 raft_consensus.cc:515] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.370272 30559 leader_election.cc:290] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 2 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:46.370829 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:46.371414 30482 raft_consensus.cc:3055] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:46.371732 30482 raft_consensus.cc:740] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:46.371701 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:46.372323 30632 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.372730 30482 consensus_queue.cc:260] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.373914 30482 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:46.379408 30632 raft_consensus.cc:2468] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 2.
I20260709 12:22:46.380306 30482 raft_consensus.cc:2468] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 2.
I20260709 12:22:46.380537 30523 leader_election.cc:304] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:46.381340 30682 raft_consensus.cc:2804] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:46.381742 30682 raft_consensus.cc:697] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Becoming Leader. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:22:46.382490 30682 consensus_queue.cc:237] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:46.388334 30379 catalog_manager.cc:5697] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 reported cstate change: term changed from 1 to 2, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:46.439452 30559 raft_consensus.cc:1275] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.440588 30681 consensus_queue.cc:1048] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.449375 30632 raft_consensus.cc:1275] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.450634 30761 consensus_queue.cc:1048] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.691876 30559 raft_consensus.cc:1275] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.693132 30761 consensus_queue.cc:1048] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.696137 30558 raft_consensus.cc:1275] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.697463 30761 consensus_queue.cc:1048] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.702322 30632 raft_consensus.cc:1275] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.703699 30761 consensus_queue.cc:1048] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.709066 30631 raft_consensus.cc:1275] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.710824 30761 consensus_queue.cc:1048] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.731714 30631 raft_consensus.cc:1275] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.732755 30703 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.739696 30559 raft_consensus.cc:1275] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.741119 30761 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.762890 30482 raft_consensus.cc:1275] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Refusing update from remote peer 58f1cc3ad9144408a7ade4db8ceffcc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.764071 30756 consensus_queue.cc:1048] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.770898 30631 raft_consensus.cc:1275] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Refusing update from remote peer 58f1cc3ad9144408a7ade4db8ceffcc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.772258 30730 consensus_queue.cc:1048] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.812090 30482 raft_consensus.cc:1275] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Refusing update from remote peer 58f1cc3ad9144408a7ade4db8ceffcc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.813254 30730 consensus_queue.cc:1048] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.820287 30631 raft_consensus.cc:1275] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Refusing update from remote peer 58f1cc3ad9144408a7ade4db8ceffcc7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.821816 30682 consensus_queue.cc:1048] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.869423 30482 raft_consensus.cc:1275] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.870659 30748 consensus_queue.cc:1048] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:46.879251 30559 raft_consensus.cc:1275] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:46.881306 30762 consensus_queue.cc:1048] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:49.065244 29845 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:22:49.065864 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:49.084689 30482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:49394
I20260709 12:22:49.085179 30482 raft_consensus.cc:606] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Received request to transfer leadership to TS a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:22:49.087425 30482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:49394
I20260709 12:22:49.087877 30482 raft_consensus.cc:606] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Received request to transfer leadership to TS a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:22:49.088798 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:49.089206 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:49.418012 30768 raft_consensus.cc:993] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b: : Instructing follower a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:22:49.418566 30761 raft_consensus.cc:1081] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Signalling peer a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:22:49.420058 30631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:44396
I20260709 12:22:49.420571 30631 raft_consensus.cc:493] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:49.420817 30631 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:49.425539 30631 raft_consensus.cc:515] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:49.427047 30631 leader_election.cc:290] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 3 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:49.427721 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:49.427973 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:49.428272 30482 raft_consensus.cc:3055] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:49.428510 30559 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:49.428620 30482 raft_consensus.cc:740] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:49.429365 30482 consensus_queue.cc:260] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:49.430294 30482 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:49.432878 30559 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 3.
I20260709 12:22:49.433858 30599 leader_election.cc:304] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:49.434542 30765 raft_consensus.cc:2804] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:49.434664 30482 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 3.
I20260709 12:22:49.435513 30765 raft_consensus.cc:697] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:49.436316 30765 consensus_queue.cc:237] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:49.442827 30382 catalog_manager.cc:5697] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 2 to 3, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 3 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:49.493918 30768 raft_consensus.cc:993] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b: : Instructing follower a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:22:49.494364 30761 raft_consensus.cc:1081] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Signalling peer a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:22:49.495652 30631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:44396
I20260709 12:22:49.496068 30631 raft_consensus.cc:493] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:49.496335 30631 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:49.500895 30631 raft_consensus.cc:515] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:49.502864 30631 leader_election.cc:290] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 3 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:22:49.503472 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:22:49.504015 30482 raft_consensus.cc:3055] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:49.503995 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:49.504422 30482 raft_consensus.cc:740] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:49.504632 30559 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:49.505158 30482 consensus_queue.cc:260] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:49.506043 30482 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:49.509179 30559 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 3.
I20260709 12:22:49.510257 30599 leader_election.cc:304] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:49.510656 30482 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 3.
I20260709 12:22:49.511317 30765 raft_consensus.cc:2804] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:49.511803 30765 raft_consensus.cc:697] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:49.512554 30765 consensus_queue.cc:237] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:49.518486 30382 catalog_manager.cc:5697] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 2 to 3, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 3 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:49.857465 30559 raft_consensus.cc:1275] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:49.858820 30765 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.869632 30482 raft_consensus.cc:1275] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:49.871893 30769 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.906500 30482 raft_consensus.cc:1275] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:49.907580 30765 consensus_queue.cc:1048] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:49.919364 30559 raft_consensus.cc:1275] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:49.921452 30765 consensus_queue.cc:1048] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:50.090045 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:50.095444 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:50.095831 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:51.096694 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:51.101950 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:51.102362 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:52.103281 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:52.108583 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:52.108985 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:53.109926 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:53.115162 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:53.115599 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:54.116472 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:54.121562 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:54.121948 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:55.123039 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:55.128365 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:55.128772 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:56.129791 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:56.135111 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:56.135520 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:57.136382 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:57.141736 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:57.142124 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:58.143016 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:58.148468 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:58.148844 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:59.177035 29845 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:22:59.187399 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:44406
I20260709 12:22:59.188009 30632 raft_consensus.cc:606] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:59.192238 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:44406
I20260709 12:22:59.192711 30632 raft_consensus.cc:606] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:59.195294 30792 raft_consensus.cc:993] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.195855 30787 raft_consensus.cc:1081] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.197573 30482 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49360
I20260709 12:22:59.198076 30482 raft_consensus.cc:493] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.198520 30482 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:59.205817 30482 raft_consensus.cc:515] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.207609 30482 leader_election.cc:290] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 4 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:59.208530 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:59.208470 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "165c70bd1d644a0b8fe47ff2183c841b" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:59.209196 30632 raft_consensus.cc:3055] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:59.209227 30559 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:59.209501 30632 raft_consensus.cc:740] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:59.210206 30632 consensus_queue.cc:260] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.211148 30632 raft_consensus.cc:3060] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:59.215593 30558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7"
dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:55974
I20260709 12:22:59.216150 30558 raft_consensus.cc:606] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:59.216293 30559 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 4.
I20260709 12:22:59.217264 30632 raft_consensus.cc:2468] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 4.
I20260709 12:22:59.217674 30450 leader_election.cc:304] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:59.218598 30786 raft_consensus.cc:2804] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:59.219803 30786 raft_consensus.cc:697] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 4 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:59.220669 30786 consensus_queue.cc:237] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.221954 30559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a"
dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:55974
I20260709 12:22:59.222451 30559 raft_consensus.cc:606] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:59.227471 30559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8"
dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
mode: GRACEFUL
new_leader_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:55974
I20260709 12:22:59.227864 30559 raft_consensus.cc:606] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Received request to transfer leadership to TS 55c82dbebacf497290d878424871882b
I20260709 12:22:59.229182 30382 catalog_manager.cc:5697] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 3 to 4, leader changed from a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 4 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.336571 30792 raft_consensus.cc:993] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.337101 30787 raft_consensus.cc:1081] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.338560 30482 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49360
I20260709 12:22:59.338997 30482 raft_consensus.cc:493] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.339282 30482 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:59.343541 30482 raft_consensus.cc:515] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.345039 30482 leader_election.cc:290] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 4 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:59.345770 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:59.345966 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:59.346263 30559 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:59.346563 30632 raft_consensus.cc:3055] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:59.346825 30632 raft_consensus.cc:740] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:22:59.347476 30632 consensus_queue.cc:260] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.348320 30632 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:59.350603 30559 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 4.
I20260709 12:22:59.351526 30450 leader_election.cc:304] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, 58f1cc3ad9144408a7ade4db8ceffcc7; no voters: 
I20260709 12:22:59.352289 30786 raft_consensus.cc:2804] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:59.352654 30786 raft_consensus.cc:697] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 4 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:59.352890 30632 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 4.
I20260709 12:22:59.353453 30786 consensus_queue.cc:237] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.359899 30382 catalog_manager.cc:5697] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 3 to 4, leader changed from a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 4 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.555588 30784 raft_consensus.cc:993] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.556066 30802 raft_consensus.cc:1081] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.557399 30482 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:22:59.557780 30482 raft_consensus.cc:493] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.557999 30482 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:59.562206 30482 raft_consensus.cc:515] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.563691 30482 leader_election.cc:290] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 3 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:59.564368 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:59.564757 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c6f64b6e3cd4f688c95b17bd68b4df8" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:59.565044 30559 raft_consensus.cc:3055] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:59.565299 30632 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:59.565388 30559 raft_consensus.cc:740] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:22:59.566181 30559 consensus_queue.cc:260] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.567967 30559 raft_consensus.cc:3060] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:59.570241 30632 raft_consensus.cc:2468] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 3.
I20260709 12:22:59.571239 30448 leader_election.cc:304] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:59.571902 30786 raft_consensus.cc:2804] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:59.572055 30559 raft_consensus.cc:2468] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 3.
I20260709 12:22:59.572306 30786 raft_consensus.cc:697] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 3 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:59.573228 30786 consensus_queue.cc:237] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.579510 30382 catalog_manager.cc:5697] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 2 to 3, leader changed from 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 3 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.584964 30784 raft_consensus.cc:993] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.585402 30802 raft_consensus.cc:1081] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.586895 30482 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:22:59.587345 30482 raft_consensus.cc:493] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.587649 30482 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:59.592564 30482 raft_consensus.cc:515] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.593847 30482 leader_election.cc:290] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 3 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:59.594666 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:59.594834 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:59.595126 30559 raft_consensus.cc:3055] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:59.595317 30632 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:59.595443 30559 raft_consensus.cc:740] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:22:59.596172 30559 consensus_queue.cc:260] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.596992 30559 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:59.599483 30632 raft_consensus.cc:2468] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 3.
I20260709 12:22:59.600410 30448 leader_election.cc:304] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:59.601045 30786 raft_consensus.cc:2804] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:59.601578 30786 raft_consensus.cc:697] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 3 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:59.601608 30559 raft_consensus.cc:2468] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 3.
I20260709 12:22:59.602577 30786 consensus_queue.cc:237] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.608771 30382 catalog_manager.cc:5697] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 2 to 3, leader changed from 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 3 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.618773 30632 raft_consensus.cc:1275] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:59.619863 30786 consensus_queue.cc:1048] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:59.628890 30559 raft_consensus.cc:1275] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:59.630808 30786 consensus_queue.cc:1048] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:59.737725 30784 raft_consensus.cc:993] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7: : Instructing follower 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.738271 30802 raft_consensus.cc:1081] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 LEADER]: Signalling peer 55c82dbebacf497290d878424871882b to start an election
I20260709 12:22:59.739727 30482 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a"
dest_uuid: "55c82dbebacf497290d878424871882b"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:22:59.740101 30482 raft_consensus.cc:493] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:59.740319 30482 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.744283 30482 raft_consensus.cc:515] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.745688 30482 leader_election.cc:290] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Requested vote from peers 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:22:59.746578 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:22:59.746925 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a" candidate_uuid: "55c82dbebacf497290d878424871882b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:22:59.747004 30632 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.747383 30559 raft_consensus.cc:3055] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:59.747596 30559 raft_consensus.cc:740] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:22:59.748195 30559 consensus_queue.cc:260] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.749022 30559 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:59.751187 30632 raft_consensus.cc:2468] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:59.752046 30448 leader_election.cc:304] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55c82dbebacf497290d878424871882b, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:22:59.752787 30786 raft_consensus.cc:2804] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:59.753212 30786 raft_consensus.cc:697] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Leader. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:22:59.753376 30559 raft_consensus.cc:2468] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55c82dbebacf497290d878424871882b in term 2.
I20260709 12:22:59.754160 30786 consensus_queue.cc:237] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:22:59.761183 30379 catalog_manager.cc:5697] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b reported cstate change: term changed from 1 to 2, leader changed from 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66) to 55c82dbebacf497290d878424871882b (127.29.37.65). New cstate: current_term: 2 leader_uuid: "55c82dbebacf497290d878424871882b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:59.788558 30559 raft_consensus.cc:1275] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:59.789655 30786 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:59.798004 30632 raft_consensus.cc:1275] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:59.800485 30812 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:00.016256 30559 raft_consensus.cc:1275] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:00.017228 30812 consensus_queue.cc:1048] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:00.025576 30631 raft_consensus.cc:1275] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:00.028304 30786 consensus_queue.cc:1048] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:00.105504 30632 raft_consensus.cc:1275] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:00.106637 30812 consensus_queue.cc:1048] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:00.115226 30559 raft_consensus.cc:1275] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:00.116729 30786 consensus_queue.cc:1048] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:00.176499 30632 raft_consensus.cc:1275] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.177644 30786 consensus_queue.cc:1048] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:00.185978 30559 raft_consensus.cc:1275] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Refusing update from remote peer 55c82dbebacf497290d878424871882b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:00.187343 30795 consensus_queue.cc:1048] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:02.263154 29845 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:23:02.263774 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:02.271217 30482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:49394
I20260709 12:23:02.271664 30482 raft_consensus.cc:606] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Received request to transfer leadership to TS a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:23:02.273833 30482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
 from {username='slave'} at 127.0.0.1:49394
I20260709 12:23:02.274248 30482 raft_consensus.cc:606] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 4 LEADER]: Received request to transfer leadership to TS 58f1cc3ad9144408a7ade4db8ceffcc7
I20260709 12:23:02.276167 30482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
 from {username='slave'} at 127.0.0.1:49394
I20260709 12:23:02.276582 30482 raft_consensus.cc:606] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 3 LEADER]: Received request to transfer leadership to TS 58f1cc3ad9144408a7ade4db8ceffcc7
I20260709 12:23:02.278426 30482 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a"
dest_uuid: "55c82dbebacf497290d878424871882b"
mode: GRACEFUL
new_leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:49394
I20260709 12:23:02.278807 30482 raft_consensus.cc:606] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Received request to transfer leadership to TS a504e6da07bd4feb8ec83d2b4ae4706c
I20260709 12:23:02.279709 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:02.280148 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:02.433168 30815 raft_consensus.cc:993] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b: : Instructing follower a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:23:02.433620 30812 raft_consensus.cc:1081] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Signalling peer a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:23:02.434901 30631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:44396
I20260709 12:23:02.435261 30631 raft_consensus.cc:493] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:02.435472 30631 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:02.439574 30631 raft_consensus.cc:515] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.441407 30631 leader_election.cc:290] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 3 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:23:02.442117 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:23:02.442492 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5af4c29d63c344c7be09a1e528d49780" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:23:02.442699 30482 raft_consensus.cc:3055] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:02.443012 30558 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:02.442998 30482 raft_consensus.cc:740] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:23:02.443879 30482 consensus_queue.cc:260] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.444950 30482 raft_consensus.cc:3060] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:02.449268 30558 raft_consensus.cc:2468] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 3.
I20260709 12:23:02.450490 30599 leader_election.cc:304] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:23:02.450742 30482 raft_consensus.cc:2468] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 3.
I20260709 12:23:02.451177 30787 raft_consensus.cc:2804] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:02.452163 30787 raft_consensus.cc:697] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:23:02.452880 30787 consensus_queue.cc:237] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.459568 30379 catalog_manager.cc:5697] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 2 to 3, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 3 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:02.538682 30815 raft_consensus.cc:993] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b: : Instructing follower 58f1cc3ad9144408a7ade4db8ceffcc7 to start an election
I20260709 12:23:02.539170 30812 raft_consensus.cc:1081] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 3 LEADER]: Signalling peer 58f1cc3ad9144408a7ade4db8ceffcc7 to start an election
I20260709 12:23:02.540462 30558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7"
dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
 from {username='slave'} at 127.0.0.1:56446
I20260709 12:23:02.540971 30558 raft_consensus.cc:493] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:02.541431 30558 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:02.545802 30558 raft_consensus.cc:515] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.547704 30558 leader_election.cc:290] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 4 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:23:02.548306 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:23:02.548691 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbb1e8ba58b466d9be480f3f2b3a3b7" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:23:02.548875 30482 raft_consensus.cc:3055] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:02.549170 30631 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:02.549160 30482 raft_consensus.cc:740] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:23:02.550063 30482 consensus_queue.cc:260] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.551137 30482 raft_consensus.cc:3060] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:02.555135 30631 raft_consensus.cc:2468] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 4.
I20260709 12:23:02.556246 30523 leader_election.cc:304] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:23:02.556802 30482 raft_consensus.cc:2468] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 4.
I20260709 12:23:02.556900 30802 raft_consensus.cc:2804] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:02.558458 30802 raft_consensus.cc:697] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 LEADER]: Becoming Leader. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:23:02.559303 30802 consensus_queue.cc:237] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.565873 30379 catalog_manager.cc:5697] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 reported cstate change: term changed from 3 to 4, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66). New cstate: current_term: 4 leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:02.682472 30815 raft_consensus.cc:993] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b: : Instructing follower a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:23:02.682955 30812 raft_consensus.cc:1081] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Signalling peer a504e6da07bd4feb8ec83d2b4ae4706c to start an election
I20260709 12:23:02.684252 30631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a"
dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
 from {username='slave'} at 127.0.0.1:44396
I20260709 12:23:02.684764 30631 raft_consensus.cc:493] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:02.685068 30631 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:02.689608 30631 raft_consensus.cc:515] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.691296 30631 leader_election.cc:290] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 3 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019)
I20260709 12:23:02.691843 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:23:02.692385 30482 raft_consensus.cc:3055] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:02.692330 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff1f64f4f9144a8eb7d5a8ca30340c5a" candidate_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
I20260709 12:23:02.692755 30482 raft_consensus.cc:740] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:23:02.692896 30558 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:02.693454 30482 consensus_queue.cc:260] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.694545 30482 raft_consensus.cc:3060] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:02.698789 30558 raft_consensus.cc:2468] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 3.
I20260709 12:23:02.699851 30599 leader_election.cc:304] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:23:02.700093 30482 raft_consensus.cc:2468] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a504e6da07bd4feb8ec83d2b4ae4706c in term 3.
I20260709 12:23:02.700605 30787 raft_consensus.cc:2804] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:02.701117 30787 raft_consensus.cc:697] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Becoming Leader. State: Replica: a504e6da07bd4feb8ec83d2b4ae4706c, State: Running, Role: LEADER
I20260709 12:23:02.701870 30787 consensus_queue.cc:237] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.708339 30379 catalog_manager.cc:5697] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c reported cstate change: term changed from 2 to 3, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67). New cstate: current_term: 3 leader_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:02.812068 30815 raft_consensus.cc:993] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b: : Instructing follower 58f1cc3ad9144408a7ade4db8ceffcc7 to start an election
I20260709 12:23:02.812527 30812 raft_consensus.cc:1081] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 4 LEADER]: Signalling peer 58f1cc3ad9144408a7ade4db8ceffcc7 to start an election
I20260709 12:23:02.813856 30558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154"
dest_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7"
 from {username='slave'} at 127.0.0.1:56446
I20260709 12:23:02.814311 30558 raft_consensus.cc:493] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:02.814544 30558 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:02.818325 30558 raft_consensus.cc:515] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.819614 30558 leader_election.cc:290] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 5 election: Requested vote from peers 55c82dbebacf497290d878424871882b (127.29.37.65:40923), a504e6da07bd4feb8ec83d2b4ae4706c (127.29.37.67:32939)
I20260709 12:23:02.820297 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "55c82dbebacf497290d878424871882b"
I20260709 12:23:02.820709 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5edd6f3bc4164863a96901c816acc154" candidate_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c"
I20260709 12:23:02.820840 30482 raft_consensus.cc:3055] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:02.821152 30482 raft_consensus.cc:740] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 55c82dbebacf497290d878424871882b, State: Running, Role: LEADER
I20260709 12:23:02.821209 30631 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:02.821796 30482 consensus_queue.cc:260] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.822615 30482 raft_consensus.cc:3060] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:02.825276 30631 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 5.
I20260709 12:23:02.826257 30523 leader_election.cc:304] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58f1cc3ad9144408a7ade4db8ceffcc7, a504e6da07bd4feb8ec83d2b4ae4706c; no voters: 
I20260709 12:23:02.826611 30482 raft_consensus.cc:2468] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58f1cc3ad9144408a7ade4db8ceffcc7 in term 5.
I20260709 12:23:02.827113 30802 raft_consensus.cc:2804] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:02.827621 30802 raft_consensus.cc:697] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 5 LEADER]: Becoming Leader. State: Replica: 58f1cc3ad9144408a7ade4db8ceffcc7, State: Running, Role: LEADER
I20260709 12:23:02.828368 30802 consensus_queue.cc:237] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } }
I20260709 12:23:02.834182 30379 catalog_manager.cc:5697] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 reported cstate change: term changed from 4 to 5, leader changed from 55c82dbebacf497290d878424871882b (127.29.37.65) to 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66). New cstate: current_term: 5 leader_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.010020 30558 raft_consensus.cc:1275] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:03.011168 30787 consensus_queue.cc:1048] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:03.021910 30482 raft_consensus.cc:1275] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:03.023307 30834 consensus_queue.cc:1048] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:03.065315 30631 raft_consensus.cc:1275] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Refusing update from remote peer 58f1cc3ad9144408a7ade4db8ceffcc7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:03.066414 30802 consensus_queue.cc:1048] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:03.074287 30482 raft_consensus.cc:1275] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Refusing update from remote peer 58f1cc3ad9144408a7ade4db8ceffcc7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:03.075724 30802 consensus_queue.cc:1048] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:03.166597 30482 raft_consensus.cc:1275] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:03.167625 30787 consensus_queue.cc:1048] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:03.170675 30558 raft_consensus.cc:1275] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Refusing update from remote peer a504e6da07bd4feb8ec83d2b4ae4706c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:03.171896 30834 consensus_queue.cc:1048] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58f1cc3ad9144408a7ade4db8ceffcc7" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:03.221658 30631 raft_consensus.cc:1275] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 5 FOLLOWER]: Refusing update from remote peer 58f1cc3ad9144408a7ade4db8ceffcc7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:03.222815 30828 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a504e6da07bd4feb8ec83d2b4ae4706c" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:03.231525 30482 raft_consensus.cc:1275] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 5 FOLLOWER]: Refusing update from remote peer 58f1cc3ad9144408a7ade4db8ceffcc7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:03.232718 30802 consensus_queue.cc:1048] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c82dbebacf497290d878424871882b" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:03.281102 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:03.287755 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:03.288190 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:23:05.100678 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800050ec0 after lost signal to thread 29848
W20260709 12:23:05.101616 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08000555a0 after lost signal to thread 30416
W20260709 12:23:05.102470 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800060680 after lost signal to thread 30508
W20260709 12:23:05.103251 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08001ee260 after lost signal to thread 30583
W20260709 12:23:05.103971 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800122580 after lost signal to thread 30657
I20260709 12:23:05.333882 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:05.340737 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:05.341150 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:06.342077 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:06.347599 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:06.347990 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:07.348831 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:07.354085 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:07.354543 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:08.355481 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:08.361123 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:08.361543 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:09.362462 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:09.368155 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:09.368548 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:10.369366 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:10.374466 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:10.374850 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:11.375895 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:11.382411 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:11.382893 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:12.383764 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:12.388926 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:12.389336 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:13.418195 29845 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:23:13.418756 29845 tablet_server.cc:179] TabletServer@127.29.37.65:0 shutting down...
I20260709 12:23:13.451468 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:23:13.452358 30522 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
I20260709 12:23:13.452440 29845 tablet_replica.cc:333] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.453928 29845 raft_consensus.cc:2243] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.454869 29845 raft_consensus.cc:2272] T ea907919202a461a824a7edbf698d237 P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:13.456626 30522 consensus_peers.cc:597] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 -> Peer 55c82dbebacf497290d878424871882b (127.29.37.65:40923): Couldn't send request to peer 55c82dbebacf497290d878424871882b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:13.458057 29845 tablet_replica.cc:333] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.458973 29845 raft_consensus.cc:2243] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.460139 29845 raft_consensus.cc:2272] T 3d5ec309913d4bcba92de3e26da3b30c P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.462687 29845 tablet_replica.cc:333] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.463398 29845 raft_consensus.cc:2243] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.463930 29845 raft_consensus.cc:2272] T 5af4c29d63c344c7be09a1e528d49780 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.466043 29845 tablet_replica.cc:333] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.466868 29845 raft_consensus.cc:2243] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.468007 29845 raft_consensus.cc:2272] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.470908 29845 tablet_replica.cc:333] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.471705 29845 raft_consensus.cc:2243] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.472273 29845 raft_consensus.cc:2272] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 55c82dbebacf497290d878424871882b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.474542 29845 tablet_replica.cc:333] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.475260 29845 raft_consensus.cc:2243] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.476109 29845 raft_consensus.cc:2272] T 165c70bd1d644a0b8fe47ff2183c841b P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.477830 29845 tablet_replica.cc:333] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.478418 29845 raft_consensus.cc:2243] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.479071 29845 raft_consensus.cc:2272] T 5edd6f3bc4164863a96901c816acc154 P 55c82dbebacf497290d878424871882b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.481452 29845 tablet_replica.cc:333] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.481941 29845 raft_consensus.cc:2243] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.482952 29845 raft_consensus.cc:2272] T 5cce934fbeed4698aa8b79694b17045b P 55c82dbebacf497290d878424871882b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.484926 29845 tablet_replica.cc:333] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.485491 29845 raft_consensus.cc:2243] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.486002 29845 raft_consensus.cc:2272] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 55c82dbebacf497290d878424871882b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.487809 29845 tablet_replica.cc:333] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b: stopping tablet replica
I20260709 12:23:13.488257 29845 raft_consensus.cc:2243] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.488758 29845 raft_consensus.cc:2272] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 55c82dbebacf497290d878424871882b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.518743 29845 tablet_server.cc:196] TabletServer@127.29.37.65:0 shutdown complete.
I20260709 12:23:13.542833 29845 tablet_server.cc:179] TabletServer@127.29.37.66:0 shutting down...
I20260709 12:23:13.566660 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:13.567355 29845 tablet_replica.cc:333] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.568023 29845 raft_consensus.cc:2243] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.568771 29845 raft_consensus.cc:2272] T ea907919202a461a824a7edbf698d237 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.570678 29845 tablet_replica.cc:333] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.571143 29845 raft_consensus.cc:2243] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.571532 29845 raft_consensus.cc:2272] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.573045 29845 tablet_replica.cc:333] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.573484 29845 raft_consensus.cc:2243] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.573959 29845 raft_consensus.cc:2272] T 5af4c29d63c344c7be09a1e528d49780 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.575632 29845 tablet_replica.cc:333] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.576171 29845 raft_consensus.cc:2243] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.576613 29845 raft_consensus.cc:2272] T 3d5ec309913d4bcba92de3e26da3b30c P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.578924 29845 tablet_replica.cc:333] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.579595 29845 raft_consensus.cc:2243] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.580060 29845 raft_consensus.cc:2272] T 165c70bd1d644a0b8fe47ff2183c841b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.581569 29845 tablet_replica.cc:333] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.582041 29845 raft_consensus.cc:2243] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.582468 29845 raft_consensus.cc:2272] T ff1f64f4f9144a8eb7d5a8ca30340c5a P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.584059 29845 tablet_replica.cc:333] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.584535 29845 raft_consensus.cc:2243] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.585204 29845 raft_consensus.cc:2272] T 5edd6f3bc4164863a96901c816acc154 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.586822 29845 tablet_replica.cc:333] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.587288 29845 raft_consensus.cc:2243] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.587684 29845 raft_consensus.cc:2272] T 5cce934fbeed4698aa8b79694b17045b P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.589272 29845 tablet_replica.cc:333] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.589721 29845 raft_consensus.cc:2243] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.590106 29845 raft_consensus.cc:2272] T 4f6de2e7a11144f3afbcabf3c9cd8802 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.591673 29845 tablet_replica.cc:333] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7: stopping tablet replica
I20260709 12:23:13.592118 29845 raft_consensus.cc:2243] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.592823 29845 raft_consensus.cc:2272] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P 58f1cc3ad9144408a7ade4db8ceffcc7 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:13.609655 30596 consensus_peers.cc:597] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c -> Peer 55c82dbebacf497290d878424871882b (127.29.37.65:40923): Couldn't send request to peer 55c82dbebacf497290d878424871882b. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:40923: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:13.623754 29845 tablet_server.cc:196] TabletServer@127.29.37.66:0 shutdown complete.
W20260709 12:23:13.643762 30599 consensus_peers.cc:597] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c -> Peer 58f1cc3ad9144408a7ade4db8ceffcc7 (127.29.37.66:43019): Couldn't send request to peer 58f1cc3ad9144408a7ade4db8ceffcc7. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:43019: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:13.645037 29845 tablet_server.cc:179] TabletServer@127.29.37.67:0 shutting down...
I20260709 12:23:13.668282 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:13.668893 29845 tablet_replica.cc:333] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.669504 29845 raft_consensus.cc:2243] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.670009 29845 raft_consensus.cc:2272] T ea907919202a461a824a7edbf698d237 P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.671890 29845 tablet_replica.cc:333] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.672403 29845 raft_consensus.cc:2243] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.672832 29845 raft_consensus.cc:2272] T 5c6f64b6e3cd4f688c95b17bd68b4df8 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.674484 29845 tablet_replica.cc:333] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.674953 29845 raft_consensus.cc:2243] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.675665 29845 raft_consensus.cc:2272] T 4f6de2e7a11144f3afbcabf3c9cd8802 P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.677351 29845 tablet_replica.cc:333] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.677874 29845 raft_consensus.cc:2243] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.678377 29845 raft_consensus.cc:2272] T bfbb1e8ba58b466d9be480f3f2b3a3b7 P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.679948 29845 tablet_replica.cc:333] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.680414 29845 raft_consensus.cc:2243] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.680799 29845 raft_consensus.cc:2272] T 5cce934fbeed4698aa8b79694b17045b P a504e6da07bd4feb8ec83d2b4ae4706c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.682510 29845 tablet_replica.cc:333] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.682981 29845 raft_consensus.cc:2243] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.683418 29845 raft_consensus.cc:2272] T 5edd6f3bc4164863a96901c816acc154 P a504e6da07bd4feb8ec83d2b4ae4706c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.684935 29845 tablet_replica.cc:333] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.685408 29845 raft_consensus.cc:2243] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.686131 29845 raft_consensus.cc:2272] T ff1f64f4f9144a8eb7d5a8ca30340c5a P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.687651 29845 tablet_replica.cc:333] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.688122 29845 raft_consensus.cc:2243] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.688509 29845 raft_consensus.cc:2272] T 165c70bd1d644a0b8fe47ff2183c841b P a504e6da07bd4feb8ec83d2b4ae4706c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.690064 29845 tablet_replica.cc:333] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.690497 29845 raft_consensus.cc:2243] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:13.690889 29845 raft_consensus.cc:2272] T 3d5ec309913d4bcba92de3e26da3b30c P a504e6da07bd4feb8ec83d2b4ae4706c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.692433 29845 tablet_replica.cc:333] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c: stopping tablet replica
I20260709 12:23:13.692848 29845 raft_consensus.cc:2243] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.693557 29845 raft_consensus.cc:2272] T 5af4c29d63c344c7be09a1e528d49780 P a504e6da07bd4feb8ec83d2b4ae4706c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.720368 29845 tablet_server.cc:196] TabletServer@127.29.37.67:0 shutdown complete.
I20260709 12:23:13.738538 29845 master.cc:562] Master@127.29.37.126:44267 shutting down...
I20260709 12:23:13.883314 29845 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:13.883929 29845 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:13.884295 29845 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8175c3d1e5934060aacf4b050f8533bd: stopping tablet replica
I20260709 12:23:13.925622 29845 master.cc:584] Master@127.29.37.126:44267 shutdown complete.
55c82dbebacf497290d878424871882b  5
58f1cc3ad9144408a7ade4db8ceffcc7  4
a504e6da07bd4feb8ec83d2b4ae4706c  1
55c82dbebacf497290d878424871882b  3
58f1cc3ad9144408a7ade4db8ceffcc7  4
a504e6da07bd4feb8ec83d2b4ae4706c  3
55c82dbebacf497290d878424871882b  8
58f1cc3ad9144408a7ade4db8ceffcc7  1
a504e6da07bd4feb8ec83d2b4ae4706c  1
55c82dbebacf497290d878424871882b  4
58f1cc3ad9144408a7ade4db8ceffcc7  3
a504e6da07bd4feb8ec83d2b4ae4706c  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (29516 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:23:13.966712 29845 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.29.37.126:44053
I20260709 12:23:13.967823 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:13.973357 30864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:13.973821 30865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:13.976060 29845 server_base.cc:1061] running on GCE node
W20260709 12:23:13.976016 30867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:13.977165 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:13.977398 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:13.977550 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599793977533 us; error 0 us; skew 500 ppm
I20260709 12:23:13.978061 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:13.980307 29845 webserver.cc:533] Webserver started at http://127.29.37.126:38099/ using document root <none> and password file <none>
I20260709 12:23:13.980786 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:13.980993 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:13.981263 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:13.982465 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/master-0-root/instance:
uuid: "0423d91b75f1428faccc573fcbf419a1"
format_stamp: "Formatted at 2026-07-09 12:23:13 on dist-test-slave-g1m6"
I20260709 12:23:13.986605 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:23:13.989624 30872 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:13.990509 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:23:13.990731 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/master-0-root
uuid: "0423d91b75f1428faccc573fcbf419a1"
format_stamp: "Formatted at 2026-07-09 12:23:13 on dist-test-slave-g1m6"
I20260709 12:23:13.990958 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:14.010514 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:14.011696 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:14.049226 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.126:44053
I20260709 12:23:14.049301 30924 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.126:44053 every 8 connection(s)
I20260709 12:23:14.053195 30925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.062291 30925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Bootstrap starting.
I20260709 12:23:14.066819 30925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.070598 30925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: No bootstrap required, opened a new log
I20260709 12:23:14.072861 30925 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423d91b75f1428faccc573fcbf419a1" member_type: VOTER }
I20260709 12:23:14.073278 30925 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:14.073535 30925 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0423d91b75f1428faccc573fcbf419a1, State: Initialized, Role: FOLLOWER
I20260709 12:23:14.074064 30925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423d91b75f1428faccc573fcbf419a1" member_type: VOTER }
I20260709 12:23:14.074581 30925 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:14.074803 30925 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:14.075078 30925 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:14.078817 30925 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423d91b75f1428faccc573fcbf419a1" member_type: VOTER }
I20260709 12:23:14.079293 30925 leader_election.cc:304] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0423d91b75f1428faccc573fcbf419a1; no voters: 
I20260709 12:23:14.080317 30925 leader_election.cc:290] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:14.080693 30928 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:14.081943 30928 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 1 LEADER]: Becoming Leader. State: Replica: 0423d91b75f1428faccc573fcbf419a1, State: Running, Role: LEADER
I20260709 12:23:14.082640 30928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423d91b75f1428faccc573fcbf419a1" member_type: VOTER }
I20260709 12:23:14.083259 30925 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:14.087402 30930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0423d91b75f1428faccc573fcbf419a1. Latest consensus state: current_term: 1 leader_uuid: "0423d91b75f1428faccc573fcbf419a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423d91b75f1428faccc573fcbf419a1" member_type: VOTER } }
I20260709 12:23:14.088235 30930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:14.092797 30929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0423d91b75f1428faccc573fcbf419a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0423d91b75f1428faccc573fcbf419a1" member_type: VOTER } }
I20260709 12:23:14.093336 30929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:14.094379 30937 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:14.101718 30937 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:14.102643 29845 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:14.109730 30937 catalog_manager.cc:1383] Generated new cluster ID: f22a37d1af2246d79bed14de3aa4ba97
I20260709 12:23:14.110003 30937 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:14.150626 30937 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:14.151847 30937 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:14.163069 30937 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Generated new TSK 0
I20260709 12:23:14.163731 30937 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:14.168953 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:14.174502 30946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.175797 30947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:14.178598 29845 server_base.cc:1061] running on GCE node
W20260709 12:23:14.178650 30949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:14.179797 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:14.179976 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:14.180145 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599794180125 us; error 0 us; skew 500 ppm
I20260709 12:23:14.180665 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:14.182778 29845 webserver.cc:533] Webserver started at http://127.29.37.65:33975/ using document root <none> and password file <none>
I20260709 12:23:14.183177 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:14.183329 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:14.183530 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:14.184484 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/instance:
uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-g1m6"
I20260709 12:23:14.188515 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:23:14.191541 30954 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.192318 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:23:14.192600 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-g1m6"
I20260709 12:23:14.192842 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:14.216988 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:14.218186 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:14.219691 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:14.221872 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:14.222036 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.222340 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:14.222508 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.260553 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:34803
I20260709 12:23:14.260665 31016 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:34803 every 8 connection(s)
I20260709 12:23:14.265295 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:14.272337 31021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.272524 31022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.277318 31024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:14.277832 29845 server_base.cc:1061] running on GCE node
I20260709 12:23:14.278172 31017 heartbeater.cc:344] Connected to a master server at 127.29.37.126:44053
I20260709 12:23:14.278587 31017 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:14.279146 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:14.279441 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:14.279440 31017 heartbeater.cc:507] Master 127.29.37.126:44053 requested a full tablet report, sending...
I20260709 12:23:14.279811 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599794279792 us; error 0 us; skew 500 ppm
I20260709 12:23:14.280570 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:14.281836 30889 ts_manager.cc:194] Registered new tserver with Master: e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:14.283108 29845 webserver.cc:533] Webserver started at http://127.29.37.66:37547/ using document root <none> and password file <none>
I20260709 12:23:14.283605 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:14.283799 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:14.283882 30889 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48508
I20260709 12:23:14.284070 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:14.285409 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/instance:
uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-g1m6"
I20260709 12:23:14.289812 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:23:14.293146 31029 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.293793 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:23:14.294091 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root
uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-g1m6"
I20260709 12:23:14.294500 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:14.311723 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:14.312783 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:14.314119 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:14.316272 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:14.316485 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.316715 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:14.316880 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.354283 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.66:45157
I20260709 12:23:14.354544 31093 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.66:45157 every 8 connection(s)
I20260709 12:23:14.358927 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:14.366189 31097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.368281 31098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.369992 31100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:14.370890 29845 server_base.cc:1061] running on GCE node
I20260709 12:23:14.371912 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:14.372133 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:14.372354 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599794372333 us; error 0 us; skew 500 ppm
I20260709 12:23:14.372591 31094 heartbeater.cc:344] Connected to a master server at 127.29.37.126:44053
I20260709 12:23:14.372951 31094 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:14.372969 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:14.373685 31094 heartbeater.cc:507] Master 127.29.37.126:44053 requested a full tablet report, sending...
I20260709 12:23:14.375628 29845 webserver.cc:533] Webserver started at http://127.29.37.67:44163/ using document root <none> and password file <none>
I20260709 12:23:14.375846 30889 ts_manager.cc:194] Registered new tserver with Master: 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:14.376157 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:14.376395 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:14.376657 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:14.377234 30889 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48512
I20260709 12:23:14.377772 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/instance:
uuid: "ce27a72886964e4287059a1d66bd5b6b"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-g1m6"
I20260709 12:23:14.381948 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:23:14.385397 31105 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.386081 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:23:14.386412 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root
uuid: "ce27a72886964e4287059a1d66bd5b6b"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-g1m6"
I20260709 12:23:14.386701 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:14.415526 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:14.416795 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:14.418273 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:14.420499 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:14.420714 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.420966 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:14.421131 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.487303 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.67:36565
I20260709 12:23:14.487417 31168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.67:36565 every 8 connection(s)
I20260709 12:23:14.504375 31169 heartbeater.cc:344] Connected to a master server at 127.29.37.126:44053
I20260709 12:23:14.504858 31169 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:14.505662 31169 heartbeater.cc:507] Master 127.29.37.126:44053 requested a full tablet report, sending...
I20260709 12:23:14.507900 30889 ts_manager.cc:194] Registered new tserver with Master: ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:14.508651 29845 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017097014s
I20260709 12:23:14.510094 30889 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48516
I20260709 12:23:14.536276 30889 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48530:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:14.542508 30889 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:14.748392 31134 tablet_service.cc:1511] Processing CreateTablet for tablet 60b6187f32774daba89763099dabeb2a (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:14.750208 31134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b6187f32774daba89763099dabeb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.755159 31131 tablet_service.cc:1511] Processing CreateTablet for tablet dd024d82eb9c473ba31252aeeb287fc0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:14.752303 31133 tablet_service.cc:1511] Processing CreateTablet for tablet 45809554a739456ea0a596484ede5ac7 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:14.754087 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 924fd2989ab145f09d211890a85a03fc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:14.756619 31133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45809554a739456ea0a596484ede5ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.757200 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924fd2989ab145f09d211890a85a03fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.759593 31130 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd1ada4571045f6ba305e207cccddcf (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:14.760445 30982 tablet_service.cc:1511] Processing CreateTablet for tablet 60b6187f32774daba89763099dabeb2a (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:14.760955 31130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd1ada4571045f6ba305e207cccddcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.761807 30982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b6187f32774daba89763099dabeb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.780082 30980 tablet_service.cc:1511] Processing CreateTablet for tablet 924fd2989ab145f09d211890a85a03fc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:14.780614 31056 tablet_service.cc:1511] Processing CreateTablet for tablet 45809554a739456ea0a596484ede5ac7 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:14.781842 30980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924fd2989ab145f09d211890a85a03fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.782151 30981 tablet_service.cc:1511] Processing CreateTablet for tablet 45809554a739456ea0a596484ede5ac7 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:14.786921 30977 tablet_service.cc:1511] Processing CreateTablet for tablet 8e81e4942d094afda44355ba1d9c7735 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:14.788149 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45809554a739456ea0a596484ede5ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.784345 30979 tablet_service.cc:1511] Processing CreateTablet for tablet dd024d82eb9c473ba31252aeeb287fc0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:14.784288 31057 tablet_service.cc:1511] Processing CreateTablet for tablet 60b6187f32774daba89763099dabeb2a (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:14.789928 30979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd024d82eb9c473ba31252aeeb287fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.786002 31129 tablet_service.cc:1511] Processing CreateTablet for tablet 8e81e4942d094afda44355ba1d9c7735 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:14.785645 30978 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd1ada4571045f6ba305e207cccddcf (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:14.766076 31131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd024d82eb9c473ba31252aeeb287fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.792634 30978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd1ada4571045f6ba305e207cccddcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.803555 30977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e81e4942d094afda44355ba1d9c7735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.782176 31056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45809554a739456ea0a596484ede5ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.810597 31057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b6187f32774daba89763099dabeb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.811630 30976 tablet_service.cc:1511] Processing CreateTablet for tablet 0d63b31721b1434ea576ac0fab297089 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:14.812902 30976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d63b31721b1434ea576ac0fab297089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.815461 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 72d9d9cc8be6411883e9f3f17ef865eb (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:14.816924 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d9d9cc8be6411883e9f3f17ef865eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.818897 31052 tablet_service.cc:1511] Processing CreateTablet for tablet 8e81e4942d094afda44355ba1d9c7735 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:14.823290 31129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e81e4942d094afda44355ba1d9c7735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.815534 31055 tablet_service.cc:1511] Processing CreateTablet for tablet 924fd2989ab145f09d211890a85a03fc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:14.827724 30975 tablet_service.cc:1511] Processing CreateTablet for tablet 7d43e79f1b224452ad15ff525d3a0bd9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:14.829103 30975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d43e79f1b224452ad15ff525d3a0bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.828979 31050 tablet_service.cc:1511] Processing CreateTablet for tablet 7d43e79f1b224452ad15ff525d3a0bd9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:14.830988 31049 tablet_service.cc:1511] Processing CreateTablet for tablet 72d9d9cc8be6411883e9f3f17ef865eb (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:14.831933 31127 tablet_service.cc:1511] Processing CreateTablet for tablet 0d63b31721b1434ea576ac0fab297089 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:14.833250 31127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d63b31721b1434ea576ac0fab297089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.832940 31048 tablet_service.cc:1511] Processing CreateTablet for tablet ff2c202e27d24d1baad8ddf5d9db4391 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:14.834810 31052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e81e4942d094afda44355ba1d9c7735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.835443 31126 tablet_service.cc:1511] Processing CreateTablet for tablet 7d43e79f1b224452ad15ff525d3a0bd9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:14.836696 31126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d43e79f1b224452ad15ff525d3a0bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.839069 31048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2c202e27d24d1baad8ddf5d9db4391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.839460 31125 tablet_service.cc:1511] Processing CreateTablet for tablet 72d9d9cc8be6411883e9f3f17ef865eb (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:14.840771 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d9d9cc8be6411883e9f3f17ef865eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.842823 31049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d9d9cc8be6411883e9f3f17ef865eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.843466 31124 tablet_service.cc:1511] Processing CreateTablet for tablet ff2c202e27d24d1baad8ddf5d9db4391 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:14.845041 31124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2c202e27d24d1baad8ddf5d9db4391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.846801 31050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d43e79f1b224452ad15ff525d3a0bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.827718 31051 tablet_service.cc:1511] Processing CreateTablet for tablet 0d63b31721b1434ea576ac0fab297089 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:14.851123 31055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 924fd2989ab145f09d211890a85a03fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.851923 31051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d63b31721b1434ea576ac0fab297089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.855419 31054 tablet_service.cc:1511] Processing CreateTablet for tablet dd024d82eb9c473ba31252aeeb287fc0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:14.856645 31054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd024d82eb9c473ba31252aeeb287fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.858911 31053 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd1ada4571045f6ba305e207cccddcf (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:14.880337 31184 tablet_bootstrap.cc:492] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:14.880527 30973 tablet_service.cc:1511] Processing CreateTablet for tablet ff2c202e27d24d1baad8ddf5d9db4391 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:14.882706 30973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2c202e27d24d1baad8ddf5d9db4391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.885694 31053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd1ada4571045f6ba305e207cccddcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.908684 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 9bb92178abcc4ef385f771bd04448a5b (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:14.910032 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb92178abcc4ef385f771bd04448a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.910964 31133 tablet_service.cc:1511] Processing CreateTablet for tablet 83f49f3c3cc44994877e69c3ff83090e (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:14.912186 31133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f49f3c3cc44994877e69c3ff83090e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.914546 31134 tablet_service.cc:1511] Processing CreateTablet for tablet 41116db50e8c4b9598abe43f5d36bf12 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:14.915130 31053 tablet_service.cc:1511] Processing CreateTablet for tablet 83f49f3c3cc44994877e69c3ff83090e (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:14.915740 31134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41116db50e8c4b9598abe43f5d36bf12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.916261 31053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f49f3c3cc44994877e69c3ff83090e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.919159 31130 tablet_service.cc:1511] Processing CreateTablet for tablet 76fe9dfd8b4c4785a354982ad7e1faa4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:14.920331 31130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76fe9dfd8b4c4785a354982ad7e1faa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.936409 31124 tablet_service.cc:1511] Processing CreateTablet for tablet 635264f51b2d4410a1da9b76606a6350 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:14.937620 31124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635264f51b2d4410a1da9b76606a6350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.940832 31056 tablet_service.cc:1511] Processing CreateTablet for tablet 41116db50e8c4b9598abe43f5d36bf12 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:14.942037 31056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41116db50e8c4b9598abe43f5d36bf12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.946017 31184 tablet_bootstrap.cc:654] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:14.947903 31053 tablet_service.cc:1511] Processing CreateTablet for tablet 76fe9dfd8b4c4785a354982ad7e1faa4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:14.949611 31129 tablet_service.cc:1511] Processing CreateTablet for tablet ba8a01bf2a514da2805eb75a7c28c412 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:14.917729 31048 tablet_service.cc:1511] Processing CreateTablet for tablet 9bb92178abcc4ef385f771bd04448a5b (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:14.955446 31186 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:14.957610 31053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76fe9dfd8b4c4785a354982ad7e1faa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.964776 31057 tablet_service.cc:1511] Processing CreateTablet for tablet ba8a01bf2a514da2805eb75a7c28c412 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:14.960000 30975 tablet_service.cc:1511] Processing CreateTablet for tablet 83f49f3c3cc44994877e69c3ff83090e (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:14.966055 30975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f49f3c3cc44994877e69c3ff83090e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.967860 30982 tablet_service.cc:1511] Processing CreateTablet for tablet 41116db50e8c4b9598abe43f5d36bf12 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:14.969132 30982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41116db50e8c4b9598abe43f5d36bf12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.971199 30978 tablet_service.cc:1511] Processing CreateTablet for tablet 9bb92178abcc4ef385f771bd04448a5b (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:14.972374 30978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb92178abcc4ef385f771bd04448a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.975184 31185 tablet_bootstrap.cc:492] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:14.985131 31057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba8a01bf2a514da2805eb75a7c28c412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.986793 31048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb92178abcc4ef385f771bd04448a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.990054 31185 tablet_bootstrap.cc:654] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.016045 30981 tablet_service.cc:1511] Processing CreateTablet for tablet ba8a01bf2a514da2805eb75a7c28c412 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:15.020608 30976 tablet_service.cc:1511] Processing CreateTablet for tablet 635264f51b2d4410a1da9b76606a6350 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:15.021072 31129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba8a01bf2a514da2805eb75a7c28c412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.021879 30976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635264f51b2d4410a1da9b76606a6350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:14.994115 31055 tablet_service.cc:1511] Processing CreateTablet for tablet e39858333670419fb20dc8adde042dd2 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:15.031688 30977 tablet_service.cc:1511] Processing CreateTablet for tablet bc9ac1b8eaf94e99a3eee16d0ca65113 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:15.036110 31051 tablet_service.cc:1511] Processing CreateTablet for tablet 2439e7934b98446ca4e7d82381e8a499 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:15.038677 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba8a01bf2a514da2805eb75a7c28c412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.041792 31051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2439e7934b98446ca4e7d82381e8a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.046515 30977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9ac1b8eaf94e99a3eee16d0ca65113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.048907 31050 tablet_service.cc:1511] Processing CreateTablet for tablet 635264f51b2d4410a1da9b76606a6350 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:15.050127 31050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635264f51b2d4410a1da9b76606a6350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.050410 30979 tablet_service.cc:1511] Processing CreateTablet for tablet 76fe9dfd8b4c4785a354982ad7e1faa4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:15.051640 30979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76fe9dfd8b4c4785a354982ad7e1faa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.057402 31054 tablet_service.cc:1511] Processing CreateTablet for tablet 4950edf9dfb9412981a0b325bf866deb (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:15.039736 30974 tablet_service.cc:1511] Processing CreateTablet for tablet e39858333670419fb20dc8adde042dd2 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:15.058931 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e39858333670419fb20dc8adde042dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.063120 31129 tablet_service.cc:1511] Processing CreateTablet for tablet e39858333670419fb20dc8adde042dd2 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:15.064291 31129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e39858333670419fb20dc8adde042dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.068583 31184 tablet_bootstrap.cc:492] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:15.069065 31184 ts_tablet_manager.cc:1403] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.189s	user 0.024s	sys 0.015s
I20260709 12:23:15.071317 31126 tablet_service.cc:1511] Processing CreateTablet for tablet 4950edf9dfb9412981a0b325bf866deb (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:15.072091 31184 raft_consensus.cc:359] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.073086 31184 raft_consensus.cc:385] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.073406 31184 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.075662 31126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4950edf9dfb9412981a0b325bf866deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.076658 31184 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.080171 31125 tablet_service.cc:1511] Processing CreateTablet for tablet a7f0812b392640ecab5f5257bb0e41ee (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:15.085161 31054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4950edf9dfb9412981a0b325bf866deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.105767 31055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e39858333670419fb20dc8adde042dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.106842 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f0812b392640ecab5f5257bb0e41ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.109880 31186 tablet_bootstrap.cc:654] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.092797 31131 tablet_service.cc:1511] Processing CreateTablet for tablet 82382d1e93d244cd8307164a5ac891b0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:15.093425 30978 tablet_service.cc:1511] Processing CreateTablet for tablet 2439e7934b98446ca4e7d82381e8a499 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:15.095588 30977 tablet_service.cc:1511] Processing CreateTablet for tablet 4950edf9dfb9412981a0b325bf866deb (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:15.095937 31134 tablet_service.cc:1511] Processing CreateTablet for tablet bc9ac1b8eaf94e99a3eee16d0ca65113 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
W20260709 12:23:15.096925 30938 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:15.096849 31132 tablet_service.cc:1511] Processing CreateTablet for tablet f93e9d74965c45a19eb84bff3d1f9825 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:15.099557 30981 tablet_service.cc:1511] Processing CreateTablet for tablet a7f0812b392640ecab5f5257bb0e41ee (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:15.122611 31134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9ac1b8eaf94e99a3eee16d0ca65113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.099931 30980 tablet_service.cc:1511] Processing CreateTablet for tablet 13959daae3c149d6ae8248d1f3ca6e59 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:15.087695 31049 tablet_service.cc:1511] Processing CreateTablet for tablet 82382d1e93d244cd8307164a5ac891b0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:15.123835 31049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82382d1e93d244cd8307164a5ac891b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.089960 31130 tablet_service.cc:1511] Processing CreateTablet for tablet 2439e7934b98446ca4e7d82381e8a499 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:15.126168 31124 tablet_service.cc:1511] Processing CreateTablet for tablet 363b01e660fa4b36aa4a5112bab1ac09 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:15.127135 31130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2439e7934b98446ca4e7d82381e8a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.121049 30978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2439e7934b98446ca4e7d82381e8a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.147632 31124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363b01e660fa4b36aa4a5112bab1ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.148914 31052 tablet_service.cc:1511] Processing CreateTablet for tablet a7f0812b392640ecab5f5257bb0e41ee (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:15.150501 31131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82382d1e93d244cd8307164a5ac891b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.151950 31048 tablet_service.cc:1511] Processing CreateTablet for tablet bc9ac1b8eaf94e99a3eee16d0ca65113 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:15.105890 30975 tablet_service.cc:1511] Processing CreateTablet for tablet f93e9d74965c45a19eb84bff3d1f9825 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:15.114774 31127 tablet_service.cc:1511] Processing CreateTablet for tablet 13959daae3c149d6ae8248d1f3ca6e59 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:15.156455 31127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13959daae3c149d6ae8248d1f3ca6e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.158624 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f0812b392640ecab5f5257bb0e41ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.159207 30976 tablet_service.cc:1511] Processing CreateTablet for tablet 82382d1e93d244cd8307164a5ac891b0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:15.159878 31053 tablet_service.cc:1511] Processing CreateTablet for tablet 13959daae3c149d6ae8248d1f3ca6e59 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:15.160487 30976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82382d1e93d244cd8307164a5ac891b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.154124 31050 tablet_service.cc:1511] Processing CreateTablet for tablet f93e9d74965c45a19eb84bff3d1f9825 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:15.114905 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 363b01e660fa4b36aa4a5112bab1ac09 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:15.163275 30977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4950edf9dfb9412981a0b325bf866deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.170472 30975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93e9d74965c45a19eb84bff3d1f9825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.171597 31133 tablet_service.cc:1511] Processing CreateTablet for tablet b1c1f8d5241540fdb4fb4daa1691260f (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:15.150545 31052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7f0812b392640ecab5f5257bb0e41ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.145933 30982 tablet_service.cc:1511] Processing CreateTablet for tablet b1c1f8d5241540fdb4fb4daa1691260f (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:15.179437 31048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9ac1b8eaf94e99a3eee16d0ca65113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.180425 31053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13959daae3c149d6ae8248d1f3ca6e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.182200 31057 tablet_service.cc:1511] Processing CreateTablet for tablet 363b01e660fa4b36aa4a5112bab1ac09 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:15.183390 31057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363b01e660fa4b36aa4a5112bab1ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.185235 30980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13959daae3c149d6ae8248d1f3ca6e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.194648 31056 tablet_service.cc:1511] Processing CreateTablet for tablet b1c1f8d5241540fdb4fb4daa1691260f (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:15.197116 31185 tablet_bootstrap.cc:492] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:15.197640 31185 ts_tablet_manager.cc:1403] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.223s	user 0.009s	sys 0.008s
I20260709 12:23:15.200630 31185 raft_consensus.cc:359] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.201818 31056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c1f8d5241540fdb4fb4daa1691260f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.202276 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93e9d74965c45a19eb84bff3d1f9825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.202895 31133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c1f8d5241540fdb4fb4daa1691260f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.202122 31185 raft_consensus.cc:385] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.206622 31050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f93e9d74965c45a19eb84bff3d1f9825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.206815 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363b01e660fa4b36aa4a5112bab1ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.206967 31169 heartbeater.cc:499] Master 127.29.37.126:44053 was elected leader, sending a full tablet report...
I20260709 12:23:15.210724 30982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c1f8d5241540fdb4fb4daa1691260f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.211265 31184 ts_tablet_manager.cc:1434] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.142s	user 0.022s	sys 0.023s
I20260709 12:23:15.212234 31184 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:15.218397 30973 tablet_service.cc:1511] Processing CreateTablet for tablet 48047b2b139146c38a821328cf7bd148 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:15.219728 30973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48047b2b139146c38a821328cf7bd148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.226157 31184 tablet_bootstrap.cc:654] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.228134 31049 tablet_service.cc:1511] Processing CreateTablet for tablet 330f8038ee4e498ebefd9d773f4e7199 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:15.203392 31185 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.239315 30979 tablet_service.cc:1511] Processing CreateTablet for tablet 330f8038ee4e498ebefd9d773f4e7199 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:15.240957 30979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 330f8038ee4e498ebefd9d773f4e7199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.242709 31049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 330f8038ee4e498ebefd9d773f4e7199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.242619 31129 tablet_service.cc:1511] Processing CreateTablet for tablet 48047b2b139146c38a821328cf7bd148 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:15.243960 31129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48047b2b139146c38a821328cf7bd148. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:15.244714 31170 tablet.cc:2406] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:23:15.248857 31055 tablet_service.cc:1511] Processing CreateTablet for tablet 48047b2b139146c38a821328cf7bd148 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:15.253072 31051 tablet_service.cc:1511] Processing CreateTablet for tablet 563f5eec53c14273959c9883d2423cc9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:15.258765 30977 tablet_service.cc:1511] Processing CreateTablet for tablet 563f5eec53c14273959c9883d2423cc9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:15.259984 30977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563f5eec53c14273959c9883d2423cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.270183 31186 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:15.270711 31186 ts_tablet_manager.cc:1403] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.316s	user 0.018s	sys 0.002s
I20260709 12:23:15.277866 31125 tablet_service.cc:1511] Processing CreateTablet for tablet 563f5eec53c14273959c9883d2423cc9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:15.278128 31126 tablet_service.cc:1511] Processing CreateTablet for tablet 330f8038ee4e498ebefd9d773f4e7199 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:15.279124 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563f5eec53c14273959c9883d2423cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.280748 31055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48047b2b139146c38a821328cf7bd148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.287413 31017 heartbeater.cc:499] Master 127.29.37.126:44053 was elected leader, sending a full tablet report...
I20260709 12:23:15.299782 30973 tablet_service.cc:1511] Processing CreateTablet for tablet e3de96a6f20142c0a00ff9ee12bcc86c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:15.301020 30973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3de96a6f20142c0a00ff9ee12bcc86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.287971 31126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 330f8038ee4e498ebefd9d773f4e7199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.306175 30980 tablet_service.cc:1511] Processing CreateTablet for tablet 0c6a2f53447447459d88ab978179c65c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:15.307484 30980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6a2f53447447459d88ab978179c65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.308552 31186 raft_consensus.cc:359] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:15.305909 31054 tablet_service.cc:1511] Processing CreateTablet for tablet e3de96a6f20142c0a00ff9ee12bcc86c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:15.309258 31186 raft_consensus.cc:385] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.309579 31186 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.316030 31053 tablet_service.cc:1511] Processing CreateTablet for tablet 0c6a2f53447447459d88ab978179c65c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:15.318521 31051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563f5eec53c14273959c9883d2423cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.319211 31054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3de96a6f20142c0a00ff9ee12bcc86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.281343 31185 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.329399 31130 tablet_service.cc:1511] Processing CreateTablet for tablet e3de96a6f20142c0a00ff9ee12bcc86c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:15.329911 30978 tablet_service.cc:1511] Processing CreateTablet for tablet d48e42d074cd496f996172cff1a0c835 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:15.330725 31130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3de96a6f20142c0a00ff9ee12bcc86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.331379 30978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d48e42d074cd496f996172cff1a0c835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.331754 31184 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:15.332247 31184 ts_tablet_manager.cc:1403] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.120s	user 0.015s	sys 0.038s
I20260709 12:23:15.335309 31053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6a2f53447447459d88ab978179c65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.336614 31052 tablet_service.cc:1511] Processing CreateTablet for tablet d48e42d074cd496f996172cff1a0c835 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:15.337862 31052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d48e42d074cd496f996172cff1a0c835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.342387 31186 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:15.348639 31184 raft_consensus.cc:359] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.349593 31184 raft_consensus.cc:385] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.355402 31185 ts_tablet_manager.cc:1434] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.157s	user 0.007s	sys 0.001s
I20260709 12:23:15.356375 31185 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:15.356117 31134 tablet_service.cc:1511] Processing CreateTablet for tablet 0c6a2f53447447459d88ab978179c65c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:15.357484 31134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6a2f53447447459d88ab978179c65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.357731 31188 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.358294 31188 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.358750 30975 tablet_service.cc:1511] Processing CreateTablet for tablet 82399103216c4d5e8d2acdbbf3cee84d (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:15.359951 30975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82399103216c4d5e8d2acdbbf3cee84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.360827 31188 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:15.361301 31131 tablet_service.cc:1511] Processing CreateTablet for tablet d48e42d074cd496f996172cff1a0c835 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:15.362977 31131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d48e42d074cd496f996172cff1a0c835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.366669 30976 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff17d7bb34443adb04e437ed0079e51 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:15.367436 31094 heartbeater.cc:499] Master 127.29.37.126:44053 was elected leader, sending a full tablet report...
I20260709 12:23:15.367906 30976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff17d7bb34443adb04e437ed0079e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.374540 31193 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.375286 31193 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:15.377878 31193 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:15.393378 30981 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea7681ba29145eba9db5bc137cf18b4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:15.399225 31056 tablet_service.cc:1511] Processing CreateTablet for tablet 82399103216c4d5e8d2acdbbf3cee84d (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:15.409152 30976 tablet_service.cc:1511] Processing CreateTablet for tablet d8e9bb54e628458a94d623c051d36c1a (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:15.416484 31056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82399103216c4d5e8d2acdbbf3cee84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.417448 31186 ts_tablet_manager.cc:1434] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.146s	user 0.005s	sys 0.002s
I20260709 12:23:15.433785 31185 tablet_bootstrap.cc:654] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.434563 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea7681ba29145eba9db5bc137cf18b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.436007 30976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e9bb54e628458a94d623c051d36c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.439610 30977 tablet_service.cc:1511] Processing CreateTablet for tablet 676d422dffb24e3cb16409a80bb28758 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:15.440848 30977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676d422dffb24e3cb16409a80bb28758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.452240 31186 tablet_bootstrap.cc:492] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:15.452157 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 4a3e0211fca8429086a4403f1ec42a59 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:15.453449 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a3e0211fca8429086a4403f1ec42a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.349987 31184 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.458376 31184 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.498728 31126 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff17d7bb34443adb04e437ed0079e51 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:15.499688 31124 tablet_service.cc:1511] Processing CreateTablet for tablet 4a3e0211fca8429086a4403f1ec42a59 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:15.501696 31129 tablet_service.cc:1511] Processing CreateTablet for tablet 676d422dffb24e3cb16409a80bb28758 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:15.500115 31126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff17d7bb34443adb04e437ed0079e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.506520 31129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676d422dffb24e3cb16409a80bb28758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.508213 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:15.511281 31132 tablet_service.cc:1511] Processing CreateTablet for tablet d8e9bb54e628458a94d623c051d36c1a (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:15.512516 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e9bb54e628458a94d623c051d36c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.512523 31054 tablet_service.cc:1511] Processing CreateTablet for tablet 4ff17d7bb34443adb04e437ed0079e51 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:15.512805 31049 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea7681ba29145eba9db5bc137cf18b4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:15.513899 31054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff17d7bb34443adb04e437ed0079e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.514827 31056 tablet_service.cc:1511] Processing CreateTablet for tablet d8e9bb54e628458a94d623c051d36c1a (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:15.516981 31052 tablet_service.cc:1511] Processing CreateTablet for tablet 4a3e0211fca8429086a4403f1ec42a59 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:15.517371 31053 tablet_service.cc:1511] Processing CreateTablet for tablet 676d422dffb24e3cb16409a80bb28758 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:15.518673 31124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a3e0211fca8429086a4403f1ec42a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.524008 31184 ts_tablet_manager.cc:1434] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.191s	user 0.003s	sys 0.007s
I20260709 12:23:15.524945 31184 tablet_bootstrap.cc:492] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:15.526707 31056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e9bb54e628458a94d623c051d36c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.527328 31053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676d422dffb24e3cb16409a80bb28758. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.527887 31133 tablet_service.cc:1511] Processing CreateTablet for tablet 7ea7681ba29145eba9db5bc137cf18b4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 12:23:15.528339 31109 leader_election.cc:343] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.529124 31133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea7681ba29145eba9db5bc137cf18b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.529727 31052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a3e0211fca8429086a4403f1ec42a59. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:15.542608 30876 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 86345.7us
I20260709 12:23:15.548416 31049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ea7681ba29145eba9db5bc137cf18b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.563189 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:23:15.564828 31033 leader_election.cc:343] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.544994 31186 tablet_bootstrap.cc:654] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.569332 31192 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.569877 31192 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.575176 31192 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:15.581616 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:15.584661 31130 tablet_service.cc:1511] Processing CreateTablet for tablet 82399103216c4d5e8d2acdbbf3cee84d (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:15.585937 31130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82399103216c4d5e8d2acdbbf3cee84d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:15.600723 31106 leader_election.cc:343] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.601209 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:15.601949 31188 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.613651 31184 tablet_bootstrap.cc:654] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.619179 30979 tablet_service.cc:1511] Processing CreateTablet for tablet 9dab2cd673924dc2a2f8622e76701947 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:15.620455 30979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dab2cd673924dc2a2f8622e76701947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.620452 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 9dab2cd673924dc2a2f8622e76701947 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:15.625825 31185 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:15.625752 31125 tablet_service.cc:1511] Processing CreateTablet for tablet f311574d640f4ff69dd8e9525a8fe8c1 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:15.626478 31185 ts_tablet_manager.cc:1403] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.270s	user 0.022s	sys 0.023s
I20260709 12:23:15.628046 30981 tablet_service.cc:1511] Processing CreateTablet for tablet f311574d640f4ff69dd8e9525a8fe8c1 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:15.629315 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f311574d640f4ff69dd8e9525a8fe8c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.630748 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dab2cd673924dc2a2f8622e76701947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.632166 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f311574d640f4ff69dd8e9525a8fe8c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.634526 31053 tablet_service.cc:1511] Processing CreateTablet for tablet 044e61a321dd4521be7f9ee796ca44dc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:15.635766 31053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044e61a321dd4521be7f9ee796ca44dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.636181 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:15.637521 31033 leader_election.cc:343] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.643793 31055 tablet_service.cc:1511] Processing CreateTablet for tablet 9dab2cd673924dc2a2f8622e76701947 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:15.645254 31055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dab2cd673924dc2a2f8622e76701947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.638100 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:15.661652 31193 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.663985 30980 tablet_service.cc:1511] Processing CreateTablet for tablet 044e61a321dd4521be7f9ee796ca44dc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:15.665251 30980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044e61a321dd4521be7f9ee796ca44dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.677225 31185 raft_consensus.cc:359] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:15.677999 31185 raft_consensus.cc:385] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.678330 31185 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
W20260709 12:23:15.679504 30876 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44173.6us
W20260709 12:23:15.678534 30873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48220us
I20260709 12:23:15.679214 31185 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:15.681756 31124 tablet_service.cc:1511] Processing CreateTablet for tablet 044e61a321dd4521be7f9ee796ca44dc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:15.682564 31185 ts_tablet_manager.cc:1434] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.056s	user 0.012s	sys 0.000s
I20260709 12:23:15.683024 31124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044e61a321dd4521be7f9ee796ca44dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.683507 31185 tablet_bootstrap.cc:492] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:15.689853 31185 tablet_bootstrap.cc:654] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.698343 31049 tablet_service.cc:1511] Processing CreateTablet for tablet f311574d640f4ff69dd8e9525a8fe8c1 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:15.699666 31049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f311574d640f4ff69dd8e9525a8fe8c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.705169 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:15.707623 30955 leader_election.cc:343] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.708206 31184 tablet_bootstrap.cc:492] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:15.708675 31184 ts_tablet_manager.cc:1403] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.184s	user 0.013s	sys 0.001s
I20260709 12:23:15.711653 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:23:15.713195 30958 leader_election.cc:343] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.713004 31052 tablet_service.cc:1511] Processing CreateTablet for tablet c6dbea78f5c04486a0fc3f17e2260e2c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:15.713676 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:15.714301 31052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dbea78f5c04486a0fc3f17e2260e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.714784 31192 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.719174 30974 tablet_service.cc:1511] Processing CreateTablet for tablet c6dbea78f5c04486a0fc3f17e2260e2c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:15.720618 31186 tablet_bootstrap.cc:492] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:15.726707 31184 raft_consensus.cc:359] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.727423 31184 raft_consensus.cc:385] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.727720 31184 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.728430 31184 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.730640 31184 ts_tablet_manager.cc:1434] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:23:15.730813 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dbea78f5c04486a0fc3f17e2260e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.731642 31184 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:15.734709 31186 ts_tablet_manager.cc:1403] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.283s	user 0.000s	sys 0.014s
I20260709 12:23:15.737834 31184 tablet_bootstrap.cc:654] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.738504 31055 tablet_service.cc:1511] Processing CreateTablet for tablet 205fb3f3a3844073a49da13542da2989 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:15.739804 31055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 205fb3f3a3844073a49da13542da2989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.742301 31125 tablet_service.cc:1511] Processing CreateTablet for tablet 205fb3f3a3844073a49da13542da2989 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:15.743507 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 205fb3f3a3844073a49da13542da2989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.735170 31132 tablet_service.cc:1511] Processing CreateTablet for tablet c6dbea78f5c04486a0fc3f17e2260e2c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:15.745615 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dbea78f5c04486a0fc3f17e2260e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.745625 31056 tablet_service.cc:1511] Processing CreateTablet for tablet 4d522233ee4c4536b4fc944d7866c4d9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:15.747265 31056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d522233ee4c4536b4fc944d7866c4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.760228 31184 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:15.760720 31184 ts_tablet_manager.cc:1403] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.015s
I20260709 12:23:15.762941 31186 raft_consensus.cc:359] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.763257 30981 tablet_service.cc:1511] Processing CreateTablet for tablet 205fb3f3a3844073a49da13542da2989 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:15.769858 31186 raft_consensus.cc:385] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.770203 31186 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.763406 31184 raft_consensus.cc:359] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:15.765179 31185 tablet_bootstrap.cc:492] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:15.770547 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 205fb3f3a3844073a49da13542da2989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.770819 31184 raft_consensus.cc:385] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.771200 31185 ts_tablet_manager.cc:1403] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.088s	user 0.011s	sys 0.002s
I20260709 12:23:15.773339 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 4d522233ee4c4536b4fc944d7866c4d9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:15.774600 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d522233ee4c4536b4fc944d7866c4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.775602 31185 raft_consensus.cc:359] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.776386 31185 raft_consensus.cc:385] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.776721 31185 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.771169 31184 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.779615 31184 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:15.782150 31184 ts_tablet_manager.cc:1434] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:23:15.783169 31184 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:15.777621 31185 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.785193 31185 ts_tablet_manager.cc:1434] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:23:15.786573 31185 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:15.786644 31186 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.789628 31125 tablet_service.cc:1511] Processing CreateTablet for tablet 4d522233ee4c4536b4fc944d7866c4d9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:15.790930 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d522233ee4c4536b4fc944d7866c4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.791955 31051 tablet_service.cc:1511] Processing CreateTablet for tablet 0afdb545bb8f485993259007a00c1841 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:15.793179 31051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0afdb545bb8f485993259007a00c1841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.796447 30979 tablet_service.cc:1511] Processing CreateTablet for tablet 0afdb545bb8f485993259007a00c1841 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:15.797400 31186 ts_tablet_manager.cc:1434] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.000s
I20260709 12:23:15.797667 30979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0afdb545bb8f485993259007a00c1841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.798303 31186 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:15.804459 31186 tablet_bootstrap.cc:654] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.806663 31185 tablet_bootstrap.cc:654] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.816417 31184 tablet_bootstrap.cc:654] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.821447 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 0afdb545bb8f485993259007a00c1841 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:15.822722 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0afdb545bb8f485993259007a00c1841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.822661 30980 tablet_service.cc:1511] Processing CreateTablet for tablet b8ba55d3e2574f7a890f7a956ba4e74f (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:15.823477 31055 tablet_service.cc:1511] Processing CreateTablet for tablet b8ba55d3e2574f7a890f7a956ba4e74f (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:15.824313 30980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ba55d3e2574f7a890f7a956ba4e74f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.824700 31055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ba55d3e2574f7a890f7a956ba4e74f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.829300 31185 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:15.830476 31185 ts_tablet_manager.cc:1403] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.044s	user 0.004s	sys 0.009s
I20260709 12:23:15.833424 31185 raft_consensus.cc:359] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.836040 31184 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:15.836606 31184 ts_tablet_manager.cc:1403] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.054s	user 0.005s	sys 0.008s
I20260709 12:23:15.839463 31184 raft_consensus.cc:359] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.840531 31184 raft_consensus.cc:385] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.840891 31184 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.841678 31184 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.840201 31185 raft_consensus.cc:385] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.842756 31185 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.844071 31185 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.848490 31124 tablet_service.cc:1511] Processing CreateTablet for tablet b8ba55d3e2574f7a890f7a956ba4e74f (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:15.849896 31124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ba55d3e2574f7a890f7a956ba4e74f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.850812 31184 ts_tablet_manager.cc:1434] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:23:15.848879 31185 ts_tablet_manager.cc:1434] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 12:23:15.860016 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 544dffe603ca4dce9d6697ea6b15c925 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:15.861058 31185 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:15.864671 31184 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:15.865222 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 544dffe603ca4dce9d6697ea6b15c925 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:15.866463 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544dffe603ca4dce9d6697ea6b15c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.861219 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544dffe603ca4dce9d6697ea6b15c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.875818 31186 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:15.876310 31186 ts_tablet_manager.cc:1403] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.078s	user 0.010s	sys 0.000s
I20260709 12:23:15.877131 31185 tablet_bootstrap.cc:654] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.879344 31051 tablet_service.cc:1511] Processing CreateTablet for tablet 544dffe603ca4dce9d6697ea6b15c925 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:15.878051 31184 tablet_bootstrap.cc:654] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.880652 31051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544dffe603ca4dce9d6697ea6b15c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.879298 31186 raft_consensus.cc:359] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.882402 31186 raft_consensus.cc:385] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.882766 31186 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.886444 30981 tablet_service.cc:1511] Processing CreateTablet for tablet 1aff3791e3404a7e92be14f04f9674ec (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:15.887662 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aff3791e3404a7e92be14f04f9674ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.887589 31052 tablet_service.cc:1511] Processing CreateTablet for tablet 1aff3791e3404a7e92be14f04f9674ec (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:15.888859 31052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aff3791e3404a7e92be14f04f9674ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.890405 31186 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.894127 31186 ts_tablet_manager.cc:1434] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:23:15.895129 31186 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:15.896310 31184 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:15.897404 31184 ts_tablet_manager.cc:1403] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.000s
I20260709 12:23:15.900420 31184 raft_consensus.cc:359] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.903429 31184 raft_consensus.cc:385] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.903777 31184 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.904681 31184 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.903139 31185 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:15.905745 31185 ts_tablet_manager.cc:1403] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.045s	user 0.006s	sys 0.010s
I20260709 12:23:15.908623 31185 raft_consensus.cc:359] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.909373 31185 raft_consensus.cc:385] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:15.909718 31185 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:15.910583 31185 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.916718 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 1aff3791e3404a7e92be14f04f9674ec (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:15.917768 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aff3791e3404a7e92be14f04f9674ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.922435 31188 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.922940 31188 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.923081 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 95980265645c4f7a8d6b201f0ae58174 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:15.924264 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95980265645c4f7a8d6b201f0ae58174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.925004 31188 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:15.925468 31188 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.928258 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:15.901546 31186 tablet_bootstrap.cc:654] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.929364 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:15.929991 31068 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
W20260709 12:23:15.930953 31109 leader_election.cc:343] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.931151 31051 tablet_service.cc:1511] Processing CreateTablet for tablet d50d1465e4b84136bf90de700a93ef2c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:15.932027 31106 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:15.932396 31051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50d1465e4b84136bf90de700a93ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.932123 31124 tablet_service.cc:1511] Processing CreateTablet for tablet d50d1465e4b84136bf90de700a93ef2c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:15.925869 31188 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.932315 31184 ts_tablet_manager.cc:1434] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.001s
I20260709 12:23:15.931154 30981 tablet_service.cc:1511] Processing CreateTablet for tablet d50d1465e4b84136bf90de700a93ef2c (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:15.934795 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50d1465e4b84136bf90de700a93ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.935271 31184 tablet_bootstrap.cc:492] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:15.935531 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:15.936996 31109 leader_election.cc:343] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.937907 31188 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:15.938656 31188 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:15.938982 31188 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.939313 31188 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:15.933378 31124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50d1465e4b84136bf90de700a93ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.922456 31185 ts_tablet_manager.cc:1434] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 12:23:15.954321 31185 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:15.955109 31184 tablet_bootstrap.cc:654] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:15.956527 31051 tablet_service.cc:1511] Processing CreateTablet for tablet 95980265645c4f7a8d6b201f0ae58174 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:15.957722 31051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95980265645c4f7a8d6b201f0ae58174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.964565 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 76399531471a4790a56d52a6ea052864 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:15.965375 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76399531471a4790a56d52a6ea052864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.969303 31052 tablet_service.cc:1511] Processing CreateTablet for tablet 76399531471a4790a56d52a6ea052864 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:15.970618 31052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76399531471a4790a56d52a6ea052864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.971480 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:15.974017 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 95980265645c4f7a8d6b201f0ae58174 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:15.975082 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95980265645c4f7a8d6b201f0ae58174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.980477 30981 tablet_service.cc:1511] Processing CreateTablet for tablet e98467ef2b2b4bab954b704025d182b4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:15.981598 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98467ef2b2b4bab954b704025d182b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:15.982613 31106 leader_election.cc:343] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:15.983131 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:15.985131 31055 tablet_service.cc:1511] Processing CreateTablet for tablet e98467ef2b2b4bab954b704025d182b4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:15.986340 31055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98467ef2b2b4bab954b704025d182b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.986907 31125 tablet_service.cc:1511] Processing CreateTablet for tablet 76399531471a4790a56d52a6ea052864 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:15.988168 31188 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:15.988446 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76399531471a4790a56d52a6ea052864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.990597 31214 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:15.997932 31186 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:15.999339 30974 tablet_service.cc:1511] Processing CreateTablet for tablet d521786b7ccd41da81f7fd0d175bf582 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:16.000519 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d521786b7ccd41da81f7fd0d175bf582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.001524 31056 tablet_service.cc:1511] Processing CreateTablet for tablet d521786b7ccd41da81f7fd0d175bf582 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:16.002754 31056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d521786b7ccd41da81f7fd0d175bf582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.002805 31188 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:15.998517 31186 ts_tablet_manager.cc:1403] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.014s
I20260709 12:23:16.007934 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:23:16.009306 31109 leader_election.cc:343] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.013302 30980 tablet_service.cc:1511] Processing CreateTablet for tablet 05394269d1a4461bb79d6f93ab25ff4e (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:16.014575 30980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05394269d1a4461bb79d6f93ab25ff4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.014856 31186 raft_consensus.cc:359] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.015393 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.015654 31186 raft_consensus.cc:385] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.015991 31068 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.016157 31186 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.017254 31186 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.021991 31186 ts_tablet_manager.cc:1434] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:23:16.020026 31184 tablet_bootstrap.cc:492] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:16.027536 31184 ts_tablet_manager.cc:1403] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.092s	user 0.010s	sys 0.011s
I20260709 12:23:16.020340 31124 tablet_service.cc:1511] Processing CreateTablet for tablet d521786b7ccd41da81f7fd0d175bf582 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:16.028824 31124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d521786b7ccd41da81f7fd0d175bf582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.014382 31051 tablet_service.cc:1511] Processing CreateTablet for tablet 05394269d1a4461bb79d6f93ab25ff4e (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:16.030375 31051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05394269d1a4461bb79d6f93ab25ff4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.030834 31184 raft_consensus.cc:359] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.033354 31184 raft_consensus.cc:385] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.033697 31184 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.034495 31184 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.035868 31049 tablet_service.cc:1511] Processing CreateTablet for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:16.022938 31068 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:16.026840 31192 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.038549 31192 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.039661 31106 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:16.040309 31129 tablet_service.cc:1511] Processing CreateTablet for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:16.040681 31192 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.041193 31192 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.041523 31129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.041599 31192 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:16.042603 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.043264 31144 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:16.043637 31192 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.044446 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 05394269d1a4461bb79d6f93ab25ff4e (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:16.045167 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.045562 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05394269d1a4461bb79d6f93ab25ff4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.019793 31185 tablet_bootstrap.cc:654] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.046160 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.047035 31144 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:16.048327 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.042246 31188 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.049575 31188 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:16.050552 31188 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.037775 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:16.051719 31192 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.053591 31192 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.053984 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.054096 31192 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.041579 31125 tablet_service.cc:1511] Processing CreateTablet for tablet e98467ef2b2b4bab954b704025d182b4 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:16.057413 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98467ef2b2b4bab954b704025d182b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.033068 31186 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.062891 31218 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.063308 31218 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.063642 31218 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.063968 31193 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.064415 31193 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.066354 31193 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.068147 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.068780 30992 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:16.070003 31184 ts_tablet_manager.cc:1434] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 12:23:16.070010 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.070921 31193 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.068130 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.071244 31184 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.071585 31144 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:16.071341 31193 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.073318 31193 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.080507 31186 tablet_bootstrap.cc:654] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.080539 30981 tablet_service.cc:1511] Processing CreateTablet for tablet b4162f35b54e4acc8abf0a1f9fc084a9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:16.081502 31193 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.080601 30888 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.089196 31053 tablet_service.cc:1511] Processing CreateTablet for tablet b4162f35b54e4acc8abf0a1f9fc084a9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:16.093501 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4162f35b54e4acc8abf0a1f9fc084a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.091920 30980 tablet_service.cc:1511] Processing CreateTablet for tablet a2c77b15766d44f585ab3cb57444a324 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:16.094282 31125 tablet_service.cc:1511] Processing CreateTablet for tablet b4162f35b54e4acc8abf0a1f9fc084a9 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:16.092247 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.105690 31067 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:16.094890 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.096155 31049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.099593 31188 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.101497 31055 tablet_service.cc:1511] Processing CreateTablet for tablet a2c77b15766d44f585ab3cb57444a324 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:16.104180 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4162f35b54e4acc8abf0a1f9fc084a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.114452 30980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2c77b15766d44f585ab3cb57444a324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.115105 31188 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:16.122691 30955 leader_election.cc:343] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:16.124059 30938 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:16.126503 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:16.127219 30992 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.129640 31055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2c77b15766d44f585ab3cb57444a324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.136688 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:16.137396 31144 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.116401 31129 tablet_service.cc:1511] Processing CreateTablet for tablet a2c77b15766d44f585ab3cb57444a324 (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:16.138725 31129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2c77b15766d44f585ab3cb57444a324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.141880 31193 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.103057 31192 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.146369 31192 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.148123 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:16.148859 31143 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:16.165905 30876 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 50260.9us
I20260709 12:23:16.168463 31185 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.168494 31186 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:16.168928 31053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4162f35b54e4acc8abf0a1f9fc084a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.169026 31185 ts_tablet_manager.cc:1403] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.215s	user 0.027s	sys 0.034s
I20260709 12:23:16.169100 31186 ts_tablet_manager.cc:1403] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.136s	user 0.013s	sys 0.017s
I20260709 12:23:16.116567 31218 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:16.175122 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.180387 31132 tablet_service.cc:1511] Processing CreateTablet for tablet 0ec17bf7d52c4cb69f4d488858fecc8b (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:16.181015 30991 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:16.181516 31132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ec17bf7d52c4cb69f4d488858fecc8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.182183 31218 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.182273 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.176808 31186 raft_consensus.cc:359] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.183630 31186 raft_consensus.cc:385] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.183727 31215 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.184162 31215 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.177053 31125 tablet_service.cc:1511] Processing CreateTablet for tablet 39b66c047b4e42afad80738552eee0dc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:16.179265 31184 tablet_bootstrap.cc:654] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.180104 31185 raft_consensus.cc:359] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.187939 31185 raft_consensus.cc:385] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.188293 31185 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.188988 31143 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:16.189165 31185 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.190243 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.191036 31218 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.191505 31218 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:16.192366 31218 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:16.166107 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48343.5us
I20260709 12:23:16.185191 31215 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.183991 31186 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.185913 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:16.185886 31125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b66c047b4e42afad80738552eee0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.196053 31142 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.210505 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.204245 31188 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:23:16.212723 31106 leader_election.cc:343] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.212759 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.210530 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.213433 31068 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.215483 31052 tablet_service.cc:1511] Processing CreateTablet for tablet 0ec17bf7d52c4cb69f4d488858fecc8b (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:16.217100 31052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ec17bf7d52c4cb69f4d488858fecc8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.217154 31049 tablet_service.cc:1511] Processing CreateTablet for tablet 39b66c047b4e42afad80738552eee0dc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:16.218649 31049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b66c047b4e42afad80738552eee0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.220525 31068 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:16.221975 31185 ts_tablet_manager.cc:1434] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.001s
I20260709 12:23:16.223006 31185 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.204988 31186 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.227730 30981 tablet_service.cc:1511] Processing CreateTablet for tablet 39b66c047b4e42afad80738552eee0dc (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:16.228770 30981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39b66c047b4e42afad80738552eee0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.209205 31142 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:16.250890 30974 tablet_service.cc:1511] Processing CreateTablet for tablet 0ec17bf7d52c4cb69f4d488858fecc8b (DEFAULT_TABLE table=test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:16.252401 30974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ec17bf7d52c4cb69f4d488858fecc8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.254474 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.255257 31192 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.246611 31193 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.262199 31184 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:16.262823 31184 ts_tablet_manager.cc:1403] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.192s	user 0.050s	sys 0.033s
I20260709 12:23:16.260851 31193 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.264720 31186 ts_tablet_manager.cc:1434] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.001s
I20260709 12:23:16.265834 31193 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.270570 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.272200 31186 tablet_bootstrap.cc:492] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.241812 31144 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:16.281225 31184 raft_consensus.cc:359] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.282961 31184 raft_consensus.cc:385] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:16.268424 30955 leader_election.cc:343] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.287024 31184 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.283931 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:16.287899 30992 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:16.288472 31193 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.283056 31192 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:16.289765 31193 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:16.289665 31192 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } }
I20260709 12:23:16.290674 31193 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.281019 30889 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:16.298302 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.298914 31142 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:16.299748 31215 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.301785 31215 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.302853 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.303263 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:16.303789 30991 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:16.304203 31106 leader_election.cc:343] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:16.308995 31033 leader_election.cc:343] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.309721 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:16.294202 31184 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.318794 31184 ts_tablet_manager.cc:1434] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 12:23:16.320148 31184 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.325452 31184 tablet_bootstrap.cc:654] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.334410 31186 tablet_bootstrap.cc:654] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.340772 31225 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.341289 31225 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.341707 31225 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.340304 31185 tablet_bootstrap.cc:654] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.356596 30991 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
W20260709 12:23:16.357388 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 47751us
I20260709 12:23:16.357869 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:16.361446 31215 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.362075 31215 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:16.363833 31215 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.367475 31186 tablet_bootstrap.cc:492] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:16.367542 31185 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.363106 30887 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:16.368060 31186 ts_tablet_manager.cc:1403] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.096s	user 0.012s	sys 0.005s
I20260709 12:23:16.368144 31185 ts_tablet_manager.cc:1403] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.145s	user 0.009s	sys 0.011s
I20260709 12:23:16.371362 31185 raft_consensus.cc:359] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.371646 31186 raft_consensus.cc:359] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.372045 31185 raft_consensus.cc:385] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.372341 31186 raft_consensus.cc:385] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.372399 31185 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.372761 31186 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.373273 31185 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.373512 31186 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.376006 31185 ts_tablet_manager.cc:1434] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:23:16.376049 31186 ts_tablet_manager.cc:1434] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:23:16.377080 31185 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.377123 31186 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.384424 31186 tablet_bootstrap.cc:654] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.385062 31185 tablet_bootstrap.cc:654] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.385474 31225 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.386206 31192 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.396484 31192 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.400552 31184 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:16.401057 31184 ts_tablet_manager.cc:1403] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.000s
I20260709 12:23:16.401822 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.403417 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.404549 31192 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:23:16.406038 30955 leader_election.cc:343] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:16.407325 30958 leader_election.cc:343] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.407749 30958 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:16.408838 31192 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.393843 30889 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.411599 31185 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.412034 31185 ts_tablet_manager.cc:1403] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.011s
I20260709 12:23:16.414628 31185 raft_consensus.cc:359] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.415257 31185 raft_consensus.cc:385] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.415516 31185 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.416131 31185 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.418077 31225 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.419181 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:16.418057 30888 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.419678 31184 raft_consensus.cc:359] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.418103 31185 ts_tablet_manager.cc:1434] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:16.420429 31184 raft_consensus.cc:385] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.420727 31184 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.420840 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:16.421248 31185 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.421371 30991 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.419795 31142 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.421401 31184 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.424280 31184 ts_tablet_manager.cc:1434] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:23:16.425128 31184 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.427258 31185 tablet_bootstrap.cc:654] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.428896 31142 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:16.428946 30991 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:16.429806 31186 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:16.430303 31186 ts_tablet_manager.cc:1403] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.020s
I20260709 12:23:16.430390 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:16.431296 31225 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.431756 31225 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:16.432657 31185 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.432531 31225 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.433063 31185 ts_tablet_manager.cc:1403] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 12:23:16.433269 31186 raft_consensus.cc:359] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.434054 31186 raft_consensus.cc:385] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.434367 31186 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.435942 31185 raft_consensus.cc:359] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.436581 31185 raft_consensus.cc:385] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.436846 31185 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.437769 31184 tablet_bootstrap.cc:654] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.437561 31185 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.439806 31185 ts_tablet_manager.cc:1434] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:16.440702 31185 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.440501 31186 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.442663 31186 ts_tablet_manager.cc:1434] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:23:16.443588 31186 tablet_bootstrap.cc:492] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.446728 31185 tablet_bootstrap.cc:654] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.449918 31186 tablet_bootstrap.cc:654] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.453775 31184 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:16.454197 31184 ts_tablet_manager.cc:1403] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 12:23:16.456570 31184 raft_consensus.cc:359] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.457178 31184 raft_consensus.cc:385] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.457438 31184 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.458210 31184 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.460444 31184 ts_tablet_manager.cc:1434] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:23:16.461412 31184 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.466636 31184 tablet_bootstrap.cc:654] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.478255 31185 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.478842 31185 ts_tablet_manager.cc:1403] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.038s	user 0.029s	sys 0.008s
I20260709 12:23:16.482048 31185 raft_consensus.cc:359] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.482960 31185 raft_consensus.cc:385] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.483330 31185 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.482769 30888 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:16.484305 31185 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.489310 31185 ts_tablet_manager.cc:1434] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:23:16.490479 31185 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.498873 31185 tablet_bootstrap.cc:654] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.510505 31185 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.511003 31185 ts_tablet_manager.cc:1403] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:23:16.510903 31192 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.511663 31192 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.513876 31185 raft_consensus.cc:359] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.514592 31185 raft_consensus.cc:385] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.514897 31185 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.515367 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.515599 31185 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:16.517289 30955 leader_election.cc:343] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.517656 31185 ts_tablet_manager.cc:1434] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:16.518707 31185 tablet_bootstrap.cc:492] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.519686 31192 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.519843 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.520967 31184 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
W20260709 12:23:16.521198 30958 leader_election.cc:343] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.521545 31184 ts_tablet_manager.cc:1403] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.060s	user 0.028s	sys 0.013s
I20260709 12:23:16.521708 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:16.522830 31192 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.525876 31184 raft_consensus.cc:359] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.526489 31184 raft_consensus.cc:385] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.526774 31184 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.527530 31184 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.528019 31186 tablet_bootstrap.cc:492] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:16.528492 31186 ts_tablet_manager.cc:1403] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.085s	user 0.043s	sys 0.036s
I20260709 12:23:16.528815 31185 tablet_bootstrap.cc:654] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.531113 31186 raft_consensus.cc:359] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.531745 31186 raft_consensus.cc:385] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.532009 31186 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
W20260709 12:23:16.532234 31109 consensus_peers.cc:597] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:16.532760 31186 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.534909 31186 ts_tablet_manager.cc:1434] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:23:16.535759 31186 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.535742 31215 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.536520 31215 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.540074 31185 tablet_bootstrap.cc:492] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.539891 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.541031 31185 ts_tablet_manager.cc:1403] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.010s
W20260709 12:23:16.541551 31106 leader_election.cc:343] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.541836 31186 tablet_bootstrap.cc:654] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.542364 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.542979 30991 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:16.544178 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:16.544059 31185 raft_consensus.cc:359] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.544907 31185 raft_consensus.cc:385] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.544976 31188 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.545329 31185 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.545363 31188 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.545888 31188 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.549242 31215 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.552996 31188 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.554629 31184 ts_tablet_manager.cc:1434] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:23:16.555624 31184 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.556154 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:16.556699 30991 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.557351 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:23:16.558741 31106 leader_election.cc:343] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.554765 31185 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.559484 31186 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:16.559856 31188 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.560032 31186 ts_tablet_manager.cc:1403] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 12:23:16.561600 31185 ts_tablet_manager.cc:1434] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 12:23:16.562803 31185 tablet_bootstrap.cc:492] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.562891 31192 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.563400 31192 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.565410 31192 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.565937 31184 tablet_bootstrap.cc:654] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.566444 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.566836 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:16.567783 30958 leader_election.cc:343] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:16.568252 30955 leader_election.cc:343] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.568161 31186 raft_consensus.cc:359] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.568831 31186 raft_consensus.cc:385] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.568759 30955 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:16.569242 31186 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.569926 31186 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.570363 31192 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.574434 31185 tablet_bootstrap.cc:654] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.576294 31186 ts_tablet_manager.cc:1434] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.002s
I20260709 12:23:16.577165 31186 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.577108 30991 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:16.578296 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:16.579339 31188 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.579809 31188 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:16.580612 31188 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.584704 31184 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:16.585376 31184 ts_tablet_manager.cc:1403] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.001s
I20260709 12:23:16.587507 31215 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.587991 31184 raft_consensus.cc:359] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.588604 31184 raft_consensus.cc:385] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.588871 31184 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.589506 31184 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.591780 31184 ts_tablet_manager.cc:1434] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:23:16.592519 31186 tablet_bootstrap.cc:654] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.595302 31184 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.597693 31185 tablet_bootstrap.cc:492] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.598627 31185 ts_tablet_manager.cc:1403] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.008s
I20260709 12:23:16.600455 30889 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.601676 31185 raft_consensus.cc:359] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.602304 31184 tablet_bootstrap.cc:654] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.602357 31185 raft_consensus.cc:385] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.602715 31185 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.603178 31186 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:16.603674 31186 ts_tablet_manager.cc:1403] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.003s
I20260709 12:23:16.606753 31186 raft_consensus.cc:359] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.607489 31186 raft_consensus.cc:385] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.607810 31186 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.608770 31184 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:16.608495 31186 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.609257 31184 ts_tablet_manager.cc:1403] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 12:23:16.610721 31186 ts_tablet_manager.cc:1434] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:16.611644 31186 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.612690 31225 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.613186 31225 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.615438 31225 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.618171 31186 tablet_bootstrap.cc:654] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.618352 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.619851 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.620296 31185 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:16.621583 31033 leader_election.cc:343] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:16.622403 31033 leader_election.cc:343] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.622794 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:16.623757 31225 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.629323 31185 ts_tablet_manager.cc:1434] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.000s
I20260709 12:23:16.630231 31185 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.633307 31184 raft_consensus.cc:359] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.634025 31184 raft_consensus.cc:385] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.634337 31184 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.637435 31192 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.637251 31184 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.637872 31192 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.639986 31192 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.642257 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.642546 31186 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:16.642894 31142 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:16.643070 31186 ts_tablet_manager.cc:1403] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.032s	user 0.022s	sys 0.007s
I20260709 12:23:16.644210 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.645046 31184 ts_tablet_manager.cc:1434] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:23:16.645761 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.646044 31186 raft_consensus.cc:359] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.646847 31186 raft_consensus.cc:385] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:16.646965 30955 leader_election.cc:343] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.647359 31184 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.647197 31186 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.645118 31192 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.648012 31192 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.648507 31192 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.648288 31186 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.650385 31186 ts_tablet_manager.cc:1434] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:16.652554 31186 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.659909 31185 tablet_bootstrap.cc:654] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.659664 31192 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.662391 31192 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.663576 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:16.664155 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.664155 31142 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.665349 31186 tablet_bootstrap.cc:654] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:16.665263 30955 leader_election.cc:343] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.666038 31192 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.666540 31192 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.670173 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.670526 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.671051 31067 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:16.672461 30955 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.673465 31226 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.673410 31225 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.671146 31144 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:16.674010 31226 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.674587 31226 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.675158 31192 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.676393 31215 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.676833 31215 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.679692 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.680119 31215 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.680433 30992 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:16.681778 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.682715 31215 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.682631 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.683221 31215 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.683578 31215 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:16.684479 31106 leader_election.cc:343] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.687184 31142 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:16.688367 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:16.689275 31192 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.692723 31184 tablet_bootstrap.cc:654] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.694790 31192 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:16.695659 31192 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.700750 31226 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.703469 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.704018 31067 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.707542 31184 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:16.707856 31188 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.707969 31184 ts_tablet_manager.cc:1403] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.000s
I20260709 12:23:16.708324 31188 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.709983 31215 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.710374 31188 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.710582 31184 raft_consensus.cc:359] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.710971 31067 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:16.711230 31184 raft_consensus.cc:385] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.711556 31184 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.712061 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.712144 30955 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.712198 31184 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.713199 31215 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.714047 31192 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.714702 31192 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:16.714787 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.715233 31226 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.716198 31192 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:16.716569 31109 leader_election.cc:343] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.716677 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:16.717295 31144 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:16.717447 31106 leader_election.cc:343] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.717681 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:16.718241 30992 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.714749 31184 ts_tablet_manager.cc:1434] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:16.721611 31184 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.726662 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.727538 31067 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:16.730947 31106 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:16.732457 31188 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.733100 31188 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.733629 31188 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.732753 31184 tablet_bootstrap.cc:654] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.740474 31144 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:16.743382 31188 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.729313 30992 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:16.750505 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:16.751519 31215 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.744408 31226 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.755944 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:23:16.757525 31109 leader_election.cc:343] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.758445 31225 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.758917 31225 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.760843 31225 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.762027 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.762869 31067 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.764699 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.765429 31143 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:16.768488 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:16.768864 31188 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.769891 31193 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.770417 31193 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.770838 31193 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.774653 31231 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.777977 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.778371 31188 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.778687 30990 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:16.778960 31188 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.784474 31192 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.791460 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:16.793411 31109 leader_election.cc:343] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.797125 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:16.798543 31106 leader_election.cc:343] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.798974 31106 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:16.800009 31268 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:16.801117 31188 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:16.803457 31067 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:16.805907 31106 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:16.806604 31231 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.807200 31231 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.810194 31231 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.807209 31188 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.811470 31188 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:16.811820 30889 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:16.812543 31188 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.814818 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.815881 31140 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:16.817808 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:16.813560 30889 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:16.821815 31231 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.822178 31231 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.822559 31231 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:16.829095 30955 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:16.829849 31192 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:16.835682 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:16.836439 30991 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:16.852555 31225 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:16.853214 31185 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.854184 31185 ts_tablet_manager.cc:1403] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.224s	user 0.017s	sys 0.064s
I20260709 12:23:16.855017 31186 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:16.855386 31186 ts_tablet_manager.cc:1403] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.204s	user 0.037s	sys 0.041s
I20260709 12:23:16.856781 31185 raft_consensus.cc:359] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.857553 31185 raft_consensus.cc:385] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.857506 31186 raft_consensus.cc:359] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.858342 31186 raft_consensus.cc:385] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.858665 31186 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.859400 31193 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.859470 31186 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.860250 31184 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:16.860663 31184 ts_tablet_manager.cc:1403] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.139s	user 0.007s	sys 0.032s
I20260709 12:23:16.862084 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:16.862649 30990 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.863152 31184 raft_consensus.cc:359] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.863864 31184 raft_consensus.cc:385] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.864141 31184 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.858029 31185 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.864909 31184 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.866110 31185 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.867715 31184 ts_tablet_manager.cc:1434] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:23:16.868099 31186 ts_tablet_manager.cc:1434] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:23:16.868929 31184 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:16.869540 31186 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:16.869827 30990 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:16.870990 31185 ts_tablet_manager.cc:1434] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:23:16.871124 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.872565 31185 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.875805 31184 tablet_bootstrap.cc:654] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.882496 31185 tablet_bootstrap.cc:654] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.895344 31186 tablet_bootstrap.cc:654] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.906116 31193 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.918918 31215 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:16.924515 31231 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.920145 31193 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.937330 31068 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.928031 31193 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:16.939020 31193 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.920617 31215 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:16.943744 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 46623.5us
I20260709 12:23:16.948688 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:16.949379 30992 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.953022 31226 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.953495 31226 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.955667 31226 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.956768 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:16.957130 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:16.957427 30992 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:16.957747 31067 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
W20260709 12:23:16.958202 30958 leader_election.cc:343] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:16.959007 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:16.960575 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:16.960861 31226 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:16.960803 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:16.961141 31144 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.961365 31226 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.961760 31226 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.968109 31144 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:16.968127 31226 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.969255 31231 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:16.970831 31193 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.977433 31267 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:23:16.970252 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:16.974493 31277 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:23:16.980453 31185 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:16.980835 31185 ts_tablet_manager.cc:1403] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.108s	user 0.036s	sys 0.032s
I20260709 12:23:16.978927 31193 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:16.982134 31193 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.983547 31276 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.986188 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:16.991685 31067 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.993803 31185 raft_consensus.cc:359] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.994505 31185 raft_consensus.cc:385] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.994833 31185 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.995497 31185 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.996963 30889 catalog_manager.cc:5697] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.998004 31185 ts_tablet_manager.cc:1434] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 12:23:16.999071 31185 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:16.987200 31141 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.003031 31226 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.014024 31184 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.014627 31184 ts_tablet_manager.cc:1403] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.146s	user 0.025s	sys 0.031s
I20260709 12:23:16.990149 31276 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:16.988181 31267 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:23:17.027107 31275 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.027747 31275 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.014876 31218 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.014734 30886 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.056696 31218 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.066758 31218 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.072053 31186 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.072495 31186 ts_tablet_manager.cc:1403] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.203s	user 0.011s	sys 0.015s
I20260709 12:23:17.073597 31185 tablet_bootstrap.cc:654] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.074146 31067 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:17.076932 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.077215 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.062696 31231 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.078331 31141 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:17.078526 31231 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:17.079885 30958 leader_election.cc:343] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.080500 31231 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.081462 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.081627 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.082545 30990 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 1: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:23:17.083663 30955 leader_election.cc:343] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.083859 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:17.084293 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:17.086738 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:17.088152 31231 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.090167 31267 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.098788 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.099179 31184 raft_consensus.cc:359] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.099751 31184 raft_consensus.cc:385] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.099826 31218 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.100095 31184 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.100551 31218 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:17.094785 31192 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.103890 31184 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.107666 31276 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.107908 31275 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.110853 31184 ts_tablet_manager.cc:1434] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.095s	user 0.005s	sys 0.008s
I20260709 12:23:17.110895 31144 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.109421 31218 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.115527 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.119686 31186 raft_consensus.cc:359] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.120323 31186 raft_consensus.cc:385] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.120599 31186 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.121037 31184 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.122125 31186 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.122921 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.115694 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.126284 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:17.128062 31186 ts_tablet_manager.cc:1434] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 12:23:17.127482 31276 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.129554 31186 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.133724 30886 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:17.134933 30938 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:17.140705 31231 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.146112 31144 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:17.130831 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.151782 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:17.152963 31231 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:17.157068 31184 tablet_bootstrap.cc:654] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.157667 31267 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.158882 31185 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.159309 31185 ts_tablet_manager.cc:1403] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.160s	user 0.010s	sys 0.009s
I20260709 12:23:17.160487 31218 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.160919 31218 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.162902 31218 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.158047 30889 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.165603 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.166610 31185 raft_consensus.cc:359] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.167244 31185 raft_consensus.cc:385] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:17.167315 30955 leader_election.cc:343] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.167528 31185 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.168295 31185 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.169260 31186 tablet_bootstrap.cc:654] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.170576 31185 ts_tablet_manager.cc:1434] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:23:17.171523 31185 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.172425 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:23:17.177153 30958 leader_election.cc:343] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.177531 31184 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.177560 30958 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:17.177996 31184 ts_tablet_manager.cc:1403] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.007s
I20260709 12:23:17.178401 31185 tablet_bootstrap.cc:654] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.178730 31218 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.180464 31184 raft_consensus.cc:359] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.181066 31184 raft_consensus.cc:385] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.181303 31184 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.181994 31184 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.184320 31184 ts_tablet_manager.cc:1434] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:17.185402 31184 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.185694 31185 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.185998 31185 ts_tablet_manager.cc:1403] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.015s	user 0.003s	sys 0.009s
I20260709 12:23:17.189018 31185 raft_consensus.cc:359] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.189847 31185 raft_consensus.cc:385] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.190230 31185 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.191188 31184 tablet_bootstrap.cc:654] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.190968 31185 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.193020 31186 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.193204 31185 ts_tablet_manager.cc:1434] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:23:17.193466 31186 ts_tablet_manager.cc:1403] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.064s	user 0.012s	sys 0.002s
I20260709 12:23:17.194056 31185 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.196163 31186 raft_consensus.cc:359] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.196846 31186 raft_consensus.cc:385] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.197191 31186 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
W20260709 12:23:17.197860 30955 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:17.197865 31186 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.200119 31186 ts_tablet_manager.cc:1434] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:23:17.200392 31185 tablet_bootstrap.cc:654] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.200982 31186 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.202921 31184 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.203361 31184 ts_tablet_manager.cc:1403] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.007s
I20260709 12:23:17.215478 31186 tablet_bootstrap.cc:654] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:17.217024 31109 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:17.219832 31184 raft_consensus.cc:359] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.220507 31184 raft_consensus.cc:385] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.220762 31184 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.221383 31184 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.223567 31267 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.223791 31186 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.223634 31184 ts_tablet_manager.cc:1434] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:23:17.224224 31186 ts_tablet_manager.cc:1403] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.001s
I20260709 12:23:17.224077 31267 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.224725 31184 tablet_bootstrap.cc:492] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.226725 31186 raft_consensus.cc:359] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.227357 31186 raft_consensus.cc:385] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.227618 31186 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.227743 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.228173 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.228359 31068 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:17.228364 31186 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.228783 30991 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:17.229172 31267 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.229703 31106 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:17.230167 31067 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.230907 31267 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.231262 31267 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.231608 31186 ts_tablet_manager.cc:1434] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:23:17.231564 31267 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.235455 31184 tablet_bootstrap.cc:654] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.239629 31192 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.239615 31267 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.242360 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:17.242904 30991 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.243238 31186 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.246618 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:17.247167 31068 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.248582 31267 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.249477 31276 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.249991 31276 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.255656 31185 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.256167 31185 ts_tablet_manager.cc:1403] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.008s
I20260709 12:23:17.255714 31186 tablet_bootstrap.cc:654] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.257714 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.258389 30992 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:17.259302 31185 raft_consensus.cc:359] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.259586 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.259939 31185 raft_consensus.cc:385] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.260267 31185 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.260468 31275 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.260843 31275 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.261158 31275 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.261793 31185 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.262120 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.262696 31141 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:17.262984 30991 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:17.264016 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.264822 31267 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.265281 31267 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:17.265642 31185 ts_tablet_manager.cc:1434] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:17.266356 31267 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.259589 31192 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:23:17.267364 31185 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.264006 31276 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.269088 31184 tablet_bootstrap.cc:492] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.269888 31184 ts_tablet_manager.cc:1403] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.003s
I20260709 12:23:17.274255 31185 tablet_bootstrap.cc:654] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.279855 31186 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.273882 31184 raft_consensus.cc:359] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.280323 31186 ts_tablet_manager.cc:1403] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.001s
I20260709 12:23:17.280318 31184 raft_consensus.cc:385] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.280936 31184 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.281659 31185 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.279328 31275 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.282090 31185 ts_tablet_manager.cc:1403] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.002s
I20260709 12:23:17.284945 31186 raft_consensus.cc:359] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.285588 31186 raft_consensus.cc:385] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.285408 31185 raft_consensus.cc:359] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.285851 31186 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.286026 31185 raft_consensus.cc:385] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.286358 31185 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.286471 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:17.286559 31186 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.286981 31185 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.287276 30991 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.288617 31186 ts_tablet_manager.cc:1434] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:23:17.288771 31185 ts_tablet_manager.cc:1434] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:17.289467 31186 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.282845 31184 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.291373 31265 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.291936 31265 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.292904 31184 ts_tablet_manager.cc:1434] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:23:17.293995 31184 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.289499 31185 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.302754 31275 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.304423 31265 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.304963 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.306406 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:17.307152 31141 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.314385 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.317991 31184 tablet_bootstrap.cc:654] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.307705 31068 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
W20260709 12:23:17.316097 31109 leader_election.cc:343] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.330708 31186 tablet_bootstrap.cc:654] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.326823 30991 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:17.333420 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.334720 31275 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.335191 31275 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:23:17.331718 31106 leader_election.cc:343] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.341384 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:17.342484 31265 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.337394 31275 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.351230 31192 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.346364 31141 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:17.358383 30885 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.368192 31185 tablet_bootstrap.cc:654] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.371186 31265 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.384902 30889 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.409165 31184 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.409659 31184 ts_tablet_manager.cc:1403] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.116s	user 0.008s	sys 0.007s
I20260709 12:23:17.412787 31184 raft_consensus.cc:359] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.413493 31184 raft_consensus.cc:385] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.413831 31184 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.415915 31184 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.419413 31184 ts_tablet_manager.cc:1434] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 12:23:17.420346 31184 tablet_bootstrap.cc:492] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.420205 30885 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.423009 31185 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.424263 31185 ts_tablet_manager.cc:1403] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.135s	user 0.007s	sys 0.020s
I20260709 12:23:17.430498 31185 raft_consensus.cc:359] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.431109 31184 tablet_bootstrap.cc:654] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.431157 31185 raft_consensus.cc:385] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.431782 30991 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.438020 31184 tablet_bootstrap.cc:492] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.438536 31184 ts_tablet_manager.cc:1403] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.002s
I20260709 12:23:17.433102 31186 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.439781 31186 ts_tablet_manager.cc:1403] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.150s	user 0.010s	sys 0.007s
I20260709 12:23:17.442526 31186 raft_consensus.cc:359] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.443264 31186 raft_consensus.cc:385] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.443522 31186 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.444188 31186 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.446064 31186 ts_tablet_manager.cc:1434] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:23:17.446122 31184 raft_consensus.cc:359] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.446769 31184 raft_consensus.cc:385] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.446980 31186 tablet_bootstrap.cc:492] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.447002 31184 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.431524 31185 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.450562 31184 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.451231 31185 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.453018 31184 ts_tablet_manager.cc:1434] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:23:17.453261 31185 ts_tablet_manager.cc:1434] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.000s
I20260709 12:23:17.453890 31184 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.458249 31185 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.460016 31184 tablet_bootstrap.cc:654] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.461365 31215 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.461833 31215 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.463990 31215 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.464954 31185 tablet_bootstrap.cc:654] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.456712 31186 tablet_bootstrap.cc:654] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.467353 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.467978 30991 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:17.469178 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.469385 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:17.470656 31106 leader_election.cc:343] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.470772 31215 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.471244 31215 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.471582 31215 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.473712 31275 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.477010 31265 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.478590 31215 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.481671 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:17.482208 30990 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.482424 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:23:17.483659 31106 leader_election.cc:343] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.486852 31215 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.489017 30990 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:17.490175 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:17.490664 31275 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.491076 31215 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.492619 31215 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:17.493673 31215 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.496057 31184 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.498509 31184 ts_tablet_manager.cc:1403] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.003s
I20260709 12:23:17.501555 31184 raft_consensus.cc:359] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.502053 31065 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.502184 31184 raft_consensus.cc:385] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.502493 31184 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.502614 31285 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.503118 31285 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.503254 31184 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.505292 31184 ts_tablet_manager.cc:1434] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:17.505262 31285 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.506274 31184 tablet_bootstrap.cc:492] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.506855 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.507453 31067 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:17.508927 30955 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.510601 31218 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.510944 31218 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.511276 31218 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.512490 31276 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.512841 31265 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.516002 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.516588 31144 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:17.518669 31184 tablet_bootstrap.cc:654] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.524843 31285 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.526834 31218 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.542578 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:17.542855 31218 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.543272 31065 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.543733 31321 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.544037 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:17.544766 31140 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.544206 31321 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.556342 31275 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:17.556164 30887 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.564973 31065 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:17.567718 30955 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.569012 31218 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.569499 31218 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:17.569900 31140 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:17.570678 31218 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.578563 31185 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.578963 31185 ts_tablet_manager.cc:1403] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.121s	user 0.016s	sys 0.003s
I20260709 12:23:17.581717 31321 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.586041 31192 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.586517 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.586513 31192 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.587236 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:17.588054 31106 leader_election.cc:343] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.588543 31192 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:23:17.589056 31109 leader_election.cc:343] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.589612 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:17.590569 31185 raft_consensus.cc:359] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.591228 31185 raft_consensus.cc:385] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.591529 31185 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.592633 31285 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.592729 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:17.594511 30955 leader_election.cc:343] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.594866 31321 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.592913 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.592206 31185 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:17.596683 30958 leader_election.cc:343] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.597101 30958 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:17.597896 31285 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.602311 31184 tablet_bootstrap.cc:492] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.602770 31184 ts_tablet_manager.cc:1403] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.097s	user 0.004s	sys 0.018s
I20260709 12:23:17.603830 31185 ts_tablet_manager.cc:1434] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 12:23:17.604513 31185 tablet_bootstrap.cc:492] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.605824 31068 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.611786 31184 raft_consensus.cc:359] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.612939 31184 raft_consensus.cc:385] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.613205 31184 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.613838 31184 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.617383 31186 tablet_bootstrap.cc:492] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.617813 31186 ts_tablet_manager.cc:1403] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.171s	user 0.014s	sys 0.009s
I20260709 12:23:17.618889 31184 ts_tablet_manager.cc:1434] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:23:17.619666 31184 tablet_bootstrap.cc:492] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.620155 31186 raft_consensus.cc:359] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.620798 31186 raft_consensus.cc:385] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.621049 31186 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.622529 31186 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.630548 31186 ts_tablet_manager.cc:1434] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.005s
I20260709 12:23:17.631945 31186 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.643508 31218 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.645869 31218 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.647863 31218 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.648932 31184 tablet_bootstrap.cc:654] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.648123 30885 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.653901 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.654315 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.658766 30955 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:17.659693 31285 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:17.663369 31186 tablet_bootstrap.cc:654] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.664477 31185 tablet_bootstrap.cc:654] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.673162 31231 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.673650 31231 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.675714 31231 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.677114 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.677758 31141 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:17.678947 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:17.679785 31231 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.680042 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.680228 31231 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:17.681325 31033 leader_election.cc:343] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.683462 31184 tablet_bootstrap.cc:492] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.680709 31231 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.683913 31184 ts_tablet_manager.cc:1403] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.000s
I20260709 12:23:17.685801 31185 tablet_bootstrap.cc:492] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.686398 31185 ts_tablet_manager.cc:1403] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.004s
I20260709 12:23:17.688407 31184 raft_consensus.cc:359] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.689054 31184 raft_consensus.cc:385] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.689344 31184 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.689145 31185 raft_consensus.cc:359] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.690047 31185 raft_consensus.cc:385] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.690058 31184 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.690582 31185 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.690862 31231 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.691565 31185 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.693783 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:17.694586 31185 ts_tablet_manager.cc:1434] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
W20260709 12:23:17.695597 31033 leader_election.cc:343] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.695827 31185 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.700588 31184 ts_tablet_manager.cc:1434] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 12:23:17.701817 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:17.702307 31185 tablet_bootstrap.cc:654] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.702353 31141 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.709493 31141 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:17.710633 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:17.711452 31193 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.711900 31193 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:17.711796 31321 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.712747 31193 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.718456 31231 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.719116 31184 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.725621 31185 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.726051 31185 ts_tablet_manager.cc:1403] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.016s
I20260709 12:23:17.729112 31185 raft_consensus.cc:359] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.729727 31185 raft_consensus.cc:385] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.730011 31185 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.730091 31186 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.730630 31186 ts_tablet_manager.cc:1403] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.099s	user 0.028s	sys 0.001s
I20260709 12:23:17.730811 31185 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.732905 31185 ts_tablet_manager.cc:1434] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:23:17.733433 31186 raft_consensus.cc:359] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.734083 31186 raft_consensus.cc:385] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.734341 31186 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.734899 31186 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.736877 31186 ts_tablet_manager.cc:1434] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:23:17.737707 31186 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.738850 31276 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.733807 31185 tablet_bootstrap.cc:492] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.747370 31184 tablet_bootstrap.cc:654] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.748101 31265 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.749575 31185 tablet_bootstrap.cc:654] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.748464 30888 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.755208 31276 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:17.759262 31186 tablet_bootstrap.cc:654] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.792362 31186 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.792797 31186 ts_tablet_manager.cc:1403] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.004s
I20260709 12:23:17.795552 31186 raft_consensus.cc:359] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.796208 31186 raft_consensus.cc:385] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.796463 31186 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.797050 31186 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.804082 31184 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.804533 31184 ts_tablet_manager.cc:1403] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.086s	user 0.014s	sys 0.005s
I20260709 12:23:17.807104 31186 ts_tablet_manager.cc:1434] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 12:23:17.806811 31184 raft_consensus.cc:359] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.807488 31184 raft_consensus.cc:385] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.807794 31184 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.807971 31186 tablet_bootstrap.cc:492] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.808526 31184 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.810807 31184 ts_tablet_manager.cc:1434] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:17.811656 31185 tablet_bootstrap.cc:492] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.811709 31184 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:17.812178 31185 ts_tablet_manager.cc:1403] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.079s	user 0.022s	sys 0.015s
I20260709 12:23:17.813663 31186 tablet_bootstrap.cc:654] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.813861 31265 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.814349 31265 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.816341 31265 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.818154 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.818811 30990 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:17.819352 31184 tablet_bootstrap.cc:654] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.821301 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.824966 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:17.826375 31106 leader_election.cc:343] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.827211 31265 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.827582 31265 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.827924 31265 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.832304 30990 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.838109 31184 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.838765 31184 ts_tablet_manager.cc:1403] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.011s
I20260709 12:23:17.841987 31184 raft_consensus.cc:359] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.842381 31185 raft_consensus.cc:359] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.842921 31184 raft_consensus.cc:385] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.843710 31184 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.843827 31265 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.843307 31185 raft_consensus.cc:385] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.844835 31185 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.845824 31185 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.847419 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:17.846675 31184 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.847802 31265 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.851617 31184 ts_tablet_manager.cc:1434] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:23:17.850188 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:17.848018 30992 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.853541 31184 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
W20260709 12:23:17.853992 31106 leader_election.cc:343] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:17.858098 31186 tablet_bootstrap.cc:492] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.849856 31185 ts_tablet_manager.cc:1434] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:23:17.858557 31186 ts_tablet_manager.cc:1403] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.011s
I20260709 12:23:17.858361 30992 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:17.859450 31185 tablet_bootstrap.cc:492] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.859593 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:17.860656 31265 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:17.861277 31265 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:17.861061 31186 raft_consensus.cc:359] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.861950 31186 raft_consensus.cc:385] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.862278 31186 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.862111 31265 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.863051 31186 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.866364 31185 tablet_bootstrap.cc:654] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.868505 31186 ts_tablet_manager.cc:1434] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:23:17.867758 31184 tablet_bootstrap.cc:654] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.869390 31186 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.874538 31285 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.875015 31285 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.876456 31186 tablet_bootstrap.cc:654] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.876926 31285 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.878391 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.879037 31140 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:17.880087 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.880862 31065 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:17.882426 30955 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.883491 31285 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.883968 31285 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.884387 31285 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.886641 31185 tablet_bootstrap.cc:492] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.887276 31185 ts_tablet_manager.cc:1403] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.004s
I20260709 12:23:17.902015 31185 raft_consensus.cc:359] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.903003 31185 raft_consensus.cc:385] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.903399 31185 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.904258 31185 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.908674 31185 ts_tablet_manager.cc:1434] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.021s	user 0.002s	sys 0.005s
I20260709 12:23:17.911770 31185 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:17.922762 31186 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:17.923285 31186 ts_tablet_manager.cc:1403] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.054s	user 0.029s	sys 0.016s
I20260709 12:23:17.927673 31185 tablet_bootstrap.cc:654] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.931432 31186 raft_consensus.cc:359] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.932289 31186 raft_consensus.cc:385] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.932617 31186 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.933357 31186 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.935519 31186 ts_tablet_manager.cc:1434] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 12:23:17.936545 31186 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:17.948537 31185 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:17.949088 31185 ts_tablet_manager.cc:1403] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.004s
I20260709 12:23:17.952782 31275 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.953428 31275 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.955968 31275 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.956640 31321 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.957278 31321 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.960916 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:17.961498 31141 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:17.961606 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.961851 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:17.962234 30990 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:17.962456 31068 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:17.963418 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:17.963467 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.964130 30992 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:17.964952 31321 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:17.962590 31285 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.968861 31109 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:17.968061 31275 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.974045 31275 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.974462 31275 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.980602 31321 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:17.981083 31321 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.985627 31186 tablet_bootstrap.cc:654] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.987759 30887 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.981492 31321 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.994002 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:17.994634 31068 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.995980 31184 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:17.997893 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:17.998024 31185 raft_consensus.cc:359] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:17.998483 31140 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:17.998726 31185 raft_consensus.cc:385] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.999053 31185 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.999864 31285 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:17.999686 31185 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.001752 31185 ts_tablet_manager.cc:1434] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.006s
I20260709 12:23:18.002619 31185 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.005937 31184 ts_tablet_manager.cc:1403] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.153s	user 0.031s	sys 0.033s
I20260709 12:23:18.008291 31185 tablet_bootstrap.cc:654] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.008411 31184 raft_consensus.cc:359] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.009083 31184 raft_consensus.cc:385] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.009351 31184 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.012980 31185 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:18.013393 31185 ts_tablet_manager.cc:1403] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:23:18.013875 31140 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:18.013845 31184 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.014135 31275 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.015342 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.015904 31185 raft_consensus.cc:359] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.016155 31285 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.016530 31185 raft_consensus.cc:385] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.016710 31285 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:18.016811 31185 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.016918 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.017513 30992 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.017587 31185 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.017658 31285 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.019767 31184 ts_tablet_manager.cc:1434] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:23:18.019968 31185 ts_tablet_manager.cc:1434] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:18.020612 31184 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.020641 31185 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.023963 30992 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:18.025071 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.025882 31276 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.026373 31276 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:18.026753 31184 tablet_bootstrap.cc:654] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.026898 31185 tablet_bootstrap.cc:654] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.027163 31276 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.034328 31068 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:18.037065 31064 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.038489 31192 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.046104 31265 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.061354 31186 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:18.061749 31186 ts_tablet_manager.cc:1403] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.125s	user 0.006s	sys 0.005s
I20260709 12:23:18.074442 31265 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.079984 31186 raft_consensus.cc:359] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.080615 31186 raft_consensus.cc:385] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.080895 31186 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.081516 31186 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.083521 31186 ts_tablet_manager.cc:1434] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.004s
I20260709 12:23:18.084375 31186 tablet_bootstrap.cc:492] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.084548 31218 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.088848 31215 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.089394 31215 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.092485 31215 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.092989 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:18.094200 31106 leader_election.cc:343] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.094161 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.095063 30991 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:18.096293 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:18.097705 31265 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.098152 31265 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.098527 31265 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.106451 31192 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.106916 31192 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.108942 31192 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.110654 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.111433 31144 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:18.112979 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.112890 31265 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.114806 31192 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.115324 31192 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.115787 31192 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.118635 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.119338 31065 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:18.120546 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.121186 30991 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.127094 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:18.127847 31140 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.131724 30991 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:18.133023 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.135126 31215 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.135632 31215 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:18.136454 31215 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.137152 31275 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.065084 31321 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.142863 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:18.143496 31065 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.145030 30887 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.148597 31192 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.170110 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.170809 30991 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.170885 31321 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.175148 31363 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.199527 31363 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.202821 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.203490 30990 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:18.204602 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.205340 31267 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.205605 31285 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.205679 31267 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.206139 31267 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.206130 31285 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.176015 31265 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.179227 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:18.208856 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:18.210047 31106 leader_election.cc:343] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.194144 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:18.211126 31068 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.213124 31267 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:18.191475 30938 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:23:18.210093 30955 leader_election.cc:343] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.215945 31140 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:18.217499 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.235852 30990 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.236227 31185 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:18.236657 31185 ts_tablet_manager.cc:1403] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.216s	user 0.063s	sys 0.127s
I20260709 12:23:18.217624 31068 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:18.218353 31267 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.219237 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:18.237972 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:23:18.238188 31106 leader_election.cc:343] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.239010 31218 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.239499 31218 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:18.239640 31185 raft_consensus.cc:359] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.240336 31185 raft_consensus.cc:385] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.219681 31285 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.222790 30991 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:18.240327 31218 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.241717 31109 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.242745 31185 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.242859 30990 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:18.221747 31275 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:23:18.228799 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:18.243552 31185 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.223492 31186 tablet_bootstrap.cc:654] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.245464 31185 ts_tablet_manager.cc:1434] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:23:18.246150 31185 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.227965 31192 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.230608 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.232273 31065 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:18.195266 30886 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:18.248064 30958 leader_election.cc:343] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.248502 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:18.234118 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.242753 31267 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.249789 31267 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:18.243949 31144 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.244155 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:18.251677 31185 tablet_bootstrap.cc:654] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:18.252224 31106 leader_election.cc:343] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.255589 31185 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:18.255991 31185 ts_tablet_manager.cc:1403] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:23:18.236860 31184 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.256909 31184 ts_tablet_manager.cc:1403] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.236s	user 0.047s	sys 0.066s
I20260709 12:23:18.232648 30887 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.250639 31267 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.258479 31185 raft_consensus.cc:359] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.233644 31363 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.259116 31185 raft_consensus.cc:385] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.259407 31185 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.259233 31184 raft_consensus.cc:359] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.259904 31184 raft_consensus.cc:385] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.260185 31184 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.260032 31185 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.260919 31285 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.255913 31215 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.262651 31215 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:18.262784 31185 ts_tablet_manager.cc:1434] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:23:18.263156 30990 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.264189 31185 tablet_bootstrap.cc:492] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.263821 31184 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.264518 31215 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.265838 31184 ts_tablet_manager.cc:1434] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 12:23:18.266727 31184 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.262051 31144 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:18.273749 31184 tablet_bootstrap.cc:654] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.274958 31275 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.281674 31185 tablet_bootstrap.cc:654] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.283921 31192 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.284478 31192 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.286093 31362 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.286603 31362 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.287698 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:18.289342 30955 leader_election.cc:343] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.288673 31362 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.290105 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:18.291528 31109 leader_election.cc:343] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.291723 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:23:18.293592 30958 leader_election.cc:343] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.294423 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:18.295153 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.295493 31285 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.296588 31192 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.296794 31186 tablet_bootstrap.cc:492] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
W20260709 12:23:18.296844 31106 leader_election.cc:343] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.297361 31186 ts_tablet_manager.cc:1403] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.213s	user 0.010s	sys 0.003s
I20260709 12:23:18.297320 31106 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.298110 31362 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.298574 31363 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.304419 31185 tablet_bootstrap.cc:492] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:18.305008 31185 ts_tablet_manager.cc:1403] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.004s
I20260709 12:23:18.308354 31185 raft_consensus.cc:359] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.309250 31185 raft_consensus.cc:385] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.309580 31185 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.315101 31185 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.317489 31185 ts_tablet_manager.cc:1434] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:23:18.318357 31185 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.330156 30887 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.331048 31186 raft_consensus.cc:359] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.331681 31186 raft_consensus.cc:385] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.331852 31185 tablet_bootstrap.cc:654] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.331921 31186 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.332621 31186 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.339774 31186 ts_tablet_manager.cc:1434] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 12:23:18.339991 31065 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.340444 31184 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.340656 31186 tablet_bootstrap.cc:492] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.340894 31184 ts_tablet_manager.cc:1403] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.021s
I20260709 12:23:18.343396 31184 raft_consensus.cc:359] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.344025 31184 raft_consensus.cc:385] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.344276 31184 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.344895 31218 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.344895 31184 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.345394 31218 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.347302 31186 tablet_bootstrap.cc:654] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.347411 31218 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.349434 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.351346 31185 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:18.351778 31185 ts_tablet_manager.cc:1403] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.000s
W20260709 12:23:18.352219 30955 leader_election.cc:343] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.350909 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.353655 30888 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.355405 31184 ts_tablet_manager.cc:1434] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:23:18.355302 30888 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.356258 31184 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.354151 31185 raft_consensus.cc:359] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:18.354244 30958 leader_election.cc:343] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.357707 31185 raft_consensus.cc:385] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.357930 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:18.359232 31218 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.363919 31184 tablet_bootstrap.cc:654] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.364610 31186 tablet_bootstrap.cc:492] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:18.365044 31186 ts_tablet_manager.cc:1403] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.003s
I20260709 12:23:18.357977 31185 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.367061 31185 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.367758 31186 raft_consensus.cc:359] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.368399 31186 raft_consensus.cc:385] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.368664 31186 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.369117 31185 ts_tablet_manager.cc:1434] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:23:18.369275 31186 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.370057 31185 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.371898 31186 ts_tablet_manager.cc:1434] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:23:18.373306 31215 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.376380 31184 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.376883 31184 ts_tablet_manager.cc:1403] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.000s
I20260709 12:23:18.377128 31185 tablet_bootstrap.cc:654] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.373324 31186 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.379606 31184 raft_consensus.cc:359] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.380297 31184 raft_consensus.cc:385] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.380609 31184 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.381400 31184 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.385560 31184 ts_tablet_manager.cc:1434] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:18.387890 31184 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.398284 31184 tablet_bootstrap.cc:654] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.398823 31186 tablet_bootstrap.cc:654] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.407323 31184 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.407775 31184 ts_tablet_manager.cc:1403] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.020s	user 0.002s	sys 0.009s
I20260709 12:23:18.412591 31184 raft_consensus.cc:359] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.413368 31184 raft_consensus.cc:385] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.413633 31184 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.414259 31184 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.416350 31184 ts_tablet_manager.cc:1434] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:18.417133 31184 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.422155 31186 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:18.423790 31218 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.424404 31218 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.425753 31184 tablet_bootstrap.cc:654] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.428406 31218 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.430013 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.431213 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.433457 31186 ts_tablet_manager.cc:1403] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.012s
W20260709 12:23:18.434602 30958 leader_election.cc:343] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:18.435491 30955 leader_election.cc:343] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.435918 30955 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:18.436335 31186 raft_consensus.cc:359] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.436966 31186 raft_consensus.cc:385] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.437227 31186 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.437454 31192 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.437940 31184 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.438519 31184 ts_tablet_manager.cc:1403] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 12:23:18.437871 31186 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.440858 31186 ts_tablet_manager.cc:1434] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:23:18.441146 31184 raft_consensus.cc:359] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.441740 31186 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.441937 31184 raft_consensus.cc:385] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.442286 31184 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.443132 31184 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.446103 31184 ts_tablet_manager.cc:1434] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:18.447232 31184 tablet_bootstrap.cc:492] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.448074 31186 tablet_bootstrap.cc:654] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.450198 31321 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.450845 31321 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.452936 31321 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.454468 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.455186 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:18.455914 31106 leader_election.cc:343] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.459985 31186 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
W20260709 12:23:18.459903 31109 leader_election.cc:343] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.460453 31186 ts_tablet_manager.cc:1403] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.001s
I20260709 12:23:18.460433 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.461758 31321 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.466426 31218 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.467033 31184 tablet_bootstrap.cc:654] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.466871 31218 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.468952 31218 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.470960 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.471760 31067 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:18.472807 31186 raft_consensus.cc:359] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.473632 31186 raft_consensus.cc:385] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.473825 31321 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.473963 31186 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.474380 31321 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.474913 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.474833 31186 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.475788 31218 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.476215 31218 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.476852 31218 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.473948 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.477427 31141 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:18.476425 31321 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.478606 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:18.479983 31109 leader_election.cc:343] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.483220 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:18.484530 31106 leader_election.cc:343] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.484965 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.485750 31321 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.486143 31186 ts_tablet_manager.cc:1434] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:23:18.487043 31186 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.487071 31185 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:18.490859 31185 ts_tablet_manager.cc:1403] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.121s	user 0.024s	sys 0.024s
I20260709 12:23:18.491791 31186 tablet_bootstrap.cc:654] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.496381 31185 raft_consensus.cc:359] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.497000 31185 raft_consensus.cc:385] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.497282 31185 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.497900 31185 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.499321 31184 tablet_bootstrap.cc:492] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.499752 31185 ts_tablet_manager.cc:1434] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:23:18.499861 31184 ts_tablet_manager.cc:1403] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.011s
I20260709 12:23:18.502579 31184 raft_consensus.cc:359] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.503051 31186 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:18.503189 31184 raft_consensus.cc:385] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.503458 31186 ts_tablet_manager.cc:1403] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.017s	user 0.003s	sys 0.011s
I20260709 12:23:18.503455 31184 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.505946 31186 raft_consensus.cc:359] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.506067 31184 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.506582 31186 raft_consensus.cc:385] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.506858 31186 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.507421 31186 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.507717 31276 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.508080 31184 ts_tablet_manager.cc:1434] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:18.508927 31184 tablet_bootstrap.cc:492] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.509387 31186 ts_tablet_manager.cc:1434] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 12:23:18.510301 31186 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.515475 31285 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.516739 31285 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.490192 31218 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.520416 31218 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.524248 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:18.524782 31067 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.526806 31184 tablet_bootstrap.cc:654] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.527374 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:18.527920 31140 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.533357 31186 tablet_bootstrap.cc:654] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.538493 31140 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:18.542529 30958 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.543710 31192 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.544395 31192 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:18.545255 31192 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.546185 31193 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.548416 31193 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.548843 31193 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.552537 31193 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.554718 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.555482 30991 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 1: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:23:18.554117 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.560993 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.561133 31067 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:18.562071 31193 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.567423 31185 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.588900 31186 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:18.589422 31186 ts_tablet_manager.cc:1403] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.079s	user 0.016s	sys 0.014s
I20260709 12:23:18.602835 30888 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.612710 31215 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.613199 31215 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.615209 31215 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.617149 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.617364 31186 raft_consensus.cc:359] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.617779 30991 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:18.618175 31186 raft_consensus.cc:385] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.618522 31186 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.618827 31321 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.619040 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.619540 31186 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.619961 31267 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.620131 31321 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.620604 31321 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.620417 31267 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.620990 31321 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.621922 31186 ts_tablet_manager.cc:1434] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.004s
I20260709 12:23:18.622442 31267 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.626909 31186 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.627825 31184 tablet_bootstrap.cc:492] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.628283 31184 ts_tablet_manager.cc:1403] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.120s	user 0.026s	sys 0.015s
I20260709 12:23:18.628125 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:18.630278 31106 leader_election.cc:343] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.632808 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.635257 31184 raft_consensus.cc:359] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.636021 31184 raft_consensus.cc:385] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.636291 31184 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.636896 31184 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.635651 31321 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:18.637180 31109 leader_election.cc:343] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.640045 31184 ts_tablet_manager.cc:1434] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:23:18.639988 31109 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.640945 31215 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.641551 31184 tablet_bootstrap.cc:492] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.642359 31193 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.638861 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:18.643709 31106 leader_election.cc:343] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.644176 31192 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.644618 31192 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.646631 31192 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.647356 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.648241 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.648627 31321 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.648803 30992 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.648914 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:23:18.650259 31106 leader_election.cc:343] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:18.651124 30955 leader_election.cc:343] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.651481 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.652076 31186 tablet_bootstrap.cc:654] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:18.653040 30958 leader_election.cc:343] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.648813 31185 tablet_bootstrap.cc:654] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.653910 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:18.655412 31285 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.659724 30992 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:18.661356 31184 tablet_bootstrap.cc:654] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.662670 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:18.663887 31321 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.664363 31321 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:18.685566 31265 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.688297 31184 tablet_bootstrap.cc:492] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.688920 31184 ts_tablet_manager.cc:1403] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.048s	user 0.023s	sys 0.006s
W20260709 12:23:18.691450 31106 consensus_peers.cc:597] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.691565 31184 raft_consensus.cc:359] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.693514 31184 raft_consensus.cc:385] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.693904 31184 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.693549 31185 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:18.694731 31185 ts_tablet_manager.cc:1403] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.181s	user 0.007s	sys 0.016s
I20260709 12:23:18.697414 31186 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:18.698176 31186 ts_tablet_manager.cc:1403] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.001s
I20260709 12:23:18.700047 31185 raft_consensus.cc:359] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.671052 31321 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.700588 31186 raft_consensus.cc:359] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.702916 31186 raft_consensus.cc:385] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.703222 31186 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.703903 31186 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.705070 31193 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.705754 31186 ts_tablet_manager.cc:1434] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.002s
I20260709 12:23:18.705484 31193 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.700695 31185 raft_consensus.cc:385] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.706553 31185 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.700904 31184 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.709946 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.711122 31184 ts_tablet_manager.cc:1434] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
W20260709 12:23:18.711814 31033 leader_election.cc:343] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.707191 31185 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.713366 31185 ts_tablet_manager.cc:1434] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.003s
I20260709 12:23:18.714203 31185 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.716521 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.717150 31144 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:18.719774 31185 tablet_bootstrap.cc:654] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.719879 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.721285 31275 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.721655 31275 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.721990 31275 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.722260 31215 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.723058 31186 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.728550 31186 tablet_bootstrap.cc:654] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.728941 31192 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.734936 31186 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:18.735477 31186 ts_tablet_manager.cc:1403] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 12:23:18.735234 30887 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.739215 31186 raft_consensus.cc:359] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.740141 31186 raft_consensus.cc:385] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.740545 31186 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.741426 31186 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.742316 31193 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.756424 31276 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.756871 31276 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.758809 31276 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.762598 31275 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.762986 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.763566 30991 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:18.764474 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.765275 31142 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:18.765895 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.767426 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:23:18.768254 31033 leader_election.cc:343] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.711987 31184 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.771641 31363 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.775833 31231 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.776278 31231 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.776680 31231 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.781329 31186 ts_tablet_manager.cc:1434] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:23:18.782620 31186 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.783444 31231 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.784291 31363 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.785544 31231 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.787689 31363 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.788233 31363 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.790881 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.791680 31186 tablet_bootstrap.cc:654] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:18.792382 31109 leader_election.cc:343] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.798027 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.795619 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:18.799057 31144 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:18.799402 31106 leader_election.cc:343] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.799880 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.803350 31361 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.796315 31363 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.806478 31184 tablet_bootstrap.cc:654] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.809816 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.810885 30990 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.815104 31144 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:18.805956 31275 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.818068 30990 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:18.837594 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:18.838331 31144 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.837160 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.863692 31363 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.875154 31185 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:18.875737 31185 ts_tablet_manager.cc:1403] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.162s	user 0.042s	sys 0.038s
I20260709 12:23:18.864320 31231 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.868800 31144 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:18.864256 31363 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.877161 31215 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.878695 31265 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.885779 31265 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.880467 31231 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:18.893936 31186 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:18.894562 31186 ts_tablet_manager.cc:1403] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.112s	user 0.013s	sys 0.002s
I20260709 12:23:18.897606 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.899049 31193 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.899516 31193 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:18.900373 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.901064 31063 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:18.900941 31193 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.890589 31215 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.903741 31185 raft_consensus.cc:359] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.904395 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.904991 30988 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:18.904381 31185 raft_consensus.cc:385] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.906013 31185 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.899446 31363 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.914785 31185 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.916958 31285 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.917054 31231 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.918133 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.918591 31185 ts_tablet_manager.cc:1434] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 12:23:18.918848 30987 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:18.919534 31185 tablet_bootstrap.cc:492] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:18.921501 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.922407 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.923005 31321 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.923400 31321 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.923750 31321 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.924327 31362 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.924713 31362 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.925047 31362 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.905542 31215 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.928218 31184 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:18.928741 31184 ts_tablet_manager.cc:1403] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.217s	user 0.014s	sys 0.018s
I20260709 12:23:18.930142 31265 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.933990 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.941743 31184 raft_consensus.cc:359] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.942437 31184 raft_consensus.cc:385] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.942747 31184 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.943367 31184 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.937455 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:18.950615 31226 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.951627 31321 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.950470 30886 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.953596 31321 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.952618 31186 raft_consensus.cc:359] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.953864 30886 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.955374 31184 ts_tablet_manager.cc:1434] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 12:23:18.954090 31186 raft_consensus.cc:385] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.955848 31186 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.956502 31184 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:18.956552 31186 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:18.960309 31109 leader_election.cc:343] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.961733 31186 ts_tablet_manager.cc:1434] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.001s
I20260709 12:23:18.966262 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.966806 30987 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.969269 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:18.964526 31186 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:18.957821 31362 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.978348 31362 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:18.976356 30987 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:18.983425 31185 tablet_bootstrap.cc:654] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.976754 31193 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.984146 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:18.984786 31193 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:18.985627 30992 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:18.976632 31106 leader_election.cc:343] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.988698 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:18.984740 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:18.991137 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:18.991750 31321 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.994185 31142 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
W20260709 12:23:18.992368 31106 leader_election.cc:343] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.995569 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:18.996407 31276 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.996766 31276 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.997082 31276 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.997740 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:18.998459 31063 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.986984 31193 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:18.993616 31265 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.001713 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.000435 31265 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:19.003229 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:19.007804 31265 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:19.009173 31033 leader_election.cc:343] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.014580 30992 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:19.018926 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:19.020692 31063 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:19.024155 31363 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.024070 31276 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.024672 31363 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:19.025501 31363 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.026655 31276 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.031517 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:19.032792 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:19.033339 31142 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:19.034554 31033 leader_election.cc:343] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.039956 31184 tablet_bootstrap.cc:654] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:19.040937 31106 leader_election.cc:343] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.044755 31185 tablet_bootstrap.cc:492] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:19.045163 31185 ts_tablet_manager.cc:1403] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.126s	user 0.004s	sys 0.008s
I20260709 12:23:19.046312 31142 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:19.047442 31186 tablet_bootstrap.cc:654] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.047573 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:19.048413 31231 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.048844 31231 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:19.049973 31231 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.053963 31192 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.048810 31185 raft_consensus.cc:359] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.057149 31185 raft_consensus.cc:385] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.062316 31185 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.063273 31185 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.072535 31184 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.072992 31184 ts_tablet_manager.cc:1403] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.117s	user 0.006s	sys 0.005s
I20260709 12:23:19.074081 31185 ts_tablet_manager.cc:1434] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 12:23:19.075505 31184 raft_consensus.cc:359] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.076143 31184 raft_consensus.cc:385] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.076416 31184 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.077018 31184 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.078418 31184 ts_tablet_manager.cc:1434] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:23:19.079262 31184 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.079646 31186 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:19.080029 31186 ts_tablet_manager.cc:1403] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.116s	user 0.003s	sys 0.020s
I20260709 12:23:19.081764 31185 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:19.082386 31186 raft_consensus.cc:359] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.082998 31186 raft_consensus.cc:385] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.083273 31186 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.084707 31184 tablet_bootstrap.cc:654] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.084669 31186 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.086210 31186 ts_tablet_manager.cc:1434] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:23:19.087093 31186 tablet_bootstrap.cc:492] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:19.092273 31186 tablet_bootstrap.cc:654] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.096097 31184 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.096582 31184 ts_tablet_manager.cc:1403] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.018s	user 0.001s	sys 0.013s
I20260709 12:23:19.099674 31231 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.100309 31231 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.102658 31231 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.104624 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:19.105381 31142 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:19.107028 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:19.108250 31231 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.108767 31231 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.109265 31231 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.110060 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.110718 30987 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:19.112839 31184 raft_consensus.cc:359] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.113691 31184 raft_consensus.cc:385] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.114039 31184 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.114732 31184 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.116243 31184 ts_tablet_manager.cc:1434] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.000s
I20260709 12:23:19.117048 31184 tablet_bootstrap.cc:492] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.118309 31321 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.123558 31184 tablet_bootstrap.cc:654] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.126533 30885 catalog_manager.cc:5697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.133668 31063 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.138940 31321 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.139595 31321 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.143190 31321 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:19.143399 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:19.143999 31064 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:19.145095 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:19.145392 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.145916 31321 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.146003 30991 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:19.146343 31321 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.146703 31321 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.156193 31321 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.158788 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:19.159354 30992 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.133189 31231 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.164988 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:19.165525 31185 tablet_bootstrap.cc:654] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.165848 30987 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.166523 30992 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:19.167682 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:19.168496 31363 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.169092 31363 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:19.169866 31363 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.172982 31193 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.173532 31193 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.175232 31265 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.175931 31193 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.179267 31186 tablet_bootstrap.cc:492] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:19.177174 31231 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.179795 31186 ts_tablet_manager.cc:1403] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.093s	user 0.016s	sys 0.064s
I20260709 12:23:19.185477 31185 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:19.186069 31185 ts_tablet_manager.cc:1403] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.105s	user 0.009s	sys 0.003s
I20260709 12:23:19.186555 30887 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.184957 31321 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:19.192467 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:19.198091 31144 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.200387 31186 raft_consensus.cc:359] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.201012 31186 raft_consensus.cc:385] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.201280 31186 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.201922 31186 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.194909 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:19.205541 31186 ts_tablet_manager.cc:1434] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.003s
I20260709 12:23:19.206557 31186 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:19.206763 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:19.207355 31065 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.208671 31184 tablet_bootstrap.cc:492] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.209738 31184 ts_tablet_manager.cc:1403] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.093s	user 0.014s	sys 0.012s
I20260709 12:23:19.195868 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.232390 30987 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:19.233881 31185 raft_consensus.cc:359] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.234470 31185 raft_consensus.cc:385] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.234736 31185 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.235845 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:19.236115 31144 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:19.237144 31185 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.237725 31184 raft_consensus.cc:359] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.238392 31184 raft_consensus.cc:385] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.238632 31184 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.238967 31275 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.239426 31275 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:19.239214 31184 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.240197 31275 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.241271 31184 ts_tablet_manager.cc:1434] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.000s
I20260709 12:23:19.242084 31184 tablet_bootstrap.cc:492] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.244778 31185 ts_tablet_manager.cc:1434] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.001s
I20260709 12:23:19.245642 31185 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:19.246531 31033 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:19.246070 31186 tablet_bootstrap.cc:654] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.248235 31276 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:19.234699 31065 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:19.272272 31184 tablet_bootstrap.cc:654] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.272306 30886 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.274438 30886 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.276789 31185 tablet_bootstrap.cc:654] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.278345 31186 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:19.280218 31186 ts_tablet_manager.cc:1403] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.074s	user 0.021s	sys 0.000s
I20260709 12:23:19.282763 31186 raft_consensus.cc:359] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.283675 31186 raft_consensus.cc:385] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.284320 31186 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.285317 31186 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.288715 31186 ts_tablet_manager.cc:1434] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:19.289690 31186 tablet_bootstrap.cc:492] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:19.292595 31184 tablet_bootstrap.cc:492] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.293166 31184 ts_tablet_manager.cc:1403] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.004s
I20260709 12:23:19.295778 31186 tablet_bootstrap.cc:654] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.300863 31184 raft_consensus.cc:359] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.301767 31184 raft_consensus.cc:385] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.302103 31184 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.302927 31184 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.305773 31184 ts_tablet_manager.cc:1434] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:23:19.306926 31184 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.312767 31185 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:19.313206 31185 ts_tablet_manager.cc:1403] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.028s
I20260709 12:23:19.322923 31185 raft_consensus.cc:359] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.323563 31185 raft_consensus.cc:385] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.323853 31185 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.324385 31185 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.325908 30885 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:19.332959 30938 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:19.326655 31185 ts_tablet_manager.cc:1434] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:23:19.334702 31185 tablet_bootstrap.cc:492] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:19.340090 31184 tablet_bootstrap.cc:654] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.348253 31186 tablet_bootstrap.cc:492] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:19.348695 31186 ts_tablet_manager.cc:1403] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.059s	user 0.016s	sys 0.018s
I20260709 12:23:19.351239 31186 raft_consensus.cc:359] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.351881 31186 raft_consensus.cc:385] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.352164 31186 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.353076 31186 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.355017 31186 ts_tablet_manager.cc:1434] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:23:19.355064 31185 tablet_bootstrap.cc:654] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.356876 31184 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.357357 31184 ts_tablet_manager.cc:1403] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.005s
I20260709 12:23:19.358898 31186 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:19.359954 31184 raft_consensus.cc:359] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.360628 31184 raft_consensus.cc:385] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.360929 31184 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.361744 31184 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.363837 31184 ts_tablet_manager.cc:1434] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:19.364714 31184 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.364769 31186 tablet_bootstrap.cc:654] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.374281 30992 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.378108 31186 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:19.378564 31186 ts_tablet_manager.cc:1403] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 12:23:19.381075 31275 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.381467 31186 raft_consensus.cc:359] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.382130 31186 raft_consensus.cc:385] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.382421 31186 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.383003 31186 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.384953 31186 ts_tablet_manager.cc:1434] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:19.385787 31186 tablet_bootstrap.cc:492] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:19.391022 31192 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.391393 31186 tablet_bootstrap.cc:654] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.391495 31192 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.392452 31185 tablet_bootstrap.cc:492] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:19.392941 31185 ts_tablet_manager.cc:1403] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.059s	user 0.004s	sys 0.007s
I20260709 12:23:19.395721 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:19.396175 31186 tablet_bootstrap.cc:492] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:19.396175 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:19.396664 31186 ts_tablet_manager.cc:1403] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 12:23:19.397852 30958 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:19.398684 31226 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:19.398576 31185 raft_consensus.cc:359] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.399204 31186 raft_consensus.cc:359] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.399461 31185 raft_consensus.cc:385] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.399885 31186 raft_consensus.cc:385] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.399953 31185 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.400223 31186 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.400540 31275 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.401010 31186 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.401008 31185 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.403019 31184 tablet_bootstrap.cc:654] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.403196 31186 ts_tablet_manager.cc:1434] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:19.403342 31185 ts_tablet_manager.cc:1434] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:23:19.404578 31185 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:19.397858 31192 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.409351 31276 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.409817 31276 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.410404 31185 tablet_bootstrap.cc:654] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.411755 31276 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.414590 31186 tablet_bootstrap.cc:492] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:19.417164 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:19.417759 31141 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:19.420464 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.421080 30987 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:19.421856 31226 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.422354 31226 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.422760 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:19.424341 31226 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.425051 31184 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.425498 31184 ts_tablet_manager.cc:1403] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.001s
I20260709 12:23:19.426553 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:19.427817 31064 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:19.430583 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:19.448534 31226 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.449069 31226 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.449497 31226 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.451061 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:19.451798 31144 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:19.428524 31276 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.454195 31276 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.455942 31184 raft_consensus.cc:359] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.456605 31184 raft_consensus.cc:385] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.456851 31184 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.457509 31184 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.460132 31184 ts_tablet_manager.cc:1434] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.004s
I20260709 12:23:19.461005 31184 tablet_bootstrap.cc:492] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.463336 31193 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.465021 31185 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:19.465574 31185 ts_tablet_manager.cc:1403] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.000s
I20260709 12:23:19.463845 31363 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.463853 31193 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.483933 31193 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.480221 31363 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.454622 31276 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.487252 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.488421 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:19.496882 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:19.499920 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.500536 30991 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:19.501538 31193 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.501739 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:19.505184 31361 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:19.505745 31361 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.506297 31361 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.497565 31231 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:19.497782 31363 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:19.514930 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:19.515092 31193 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.522359 31065 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:19.517112 30991 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.521584 31361 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.509903 31226 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.530776 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:19.531414 30988 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.539149 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:19.539724 31065 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.529129 31361 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:19.547931 31064 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.550860 31276 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.551129 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:19.551713 31068 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.556092 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:19.556667 31140 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.559031 31363 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.561094 31065 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:19.561717 31231 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.562522 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:19.562765 31192 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.563892 31140 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:19.548512 30988 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:19.569900 31184 tablet_bootstrap.cc:654] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.569604 31192 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.590526 31185 raft_consensus.cc:359] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.593158 31185 raft_consensus.cc:385] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.593494 31185 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.594373 31192 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.594436 31185 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.571813 31361 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.598644 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:19.574980 31321 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:23:19.593945 31068 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:19.592517 31226 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.597426 31185 ts_tablet_manager.cc:1434] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.131s	user 0.005s	sys 0.002s
I20260709 12:23:19.603355 31276 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:19.603924 31361 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:19.604452 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:19.605945 31185 tablet_bootstrap.cc:492] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:19.608588 31186 tablet_bootstrap.cc:654] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.571851 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:19.608045 31140 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.614403 31265 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.614849 31265 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.616798 31265 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:19.618081 31321 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.619962 31226 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.622143 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:19.628585 31106 leader_election.cc:343] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.636826 31226 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:19.612730 31361 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.665647 31140 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:19.674479 31226 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.677045 31065 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.681797 31443 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.660476 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:19.685204 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:19.689543 31141 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:19.700410 31184 tablet_bootstrap.cc:492] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.700883 31184 ts_tablet_manager.cc:1403] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.240s	user 0.023s	sys 0.009s
I20260709 12:23:19.691659 30991 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.719408 31363 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.724802 31363 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.727226 31363 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:19.711184 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:19.730768 31276 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:19.731419 31276 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:19.732426 31276 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.764382 31184 raft_consensus.cc:359] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.765103 31184 raft_consensus.cc:385] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.765389 31184 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.766180 31184 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:19.771674 31447 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:23:19.774359 31186 tablet_bootstrap.cc:492] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:19.779982 31363 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.780453 31185 tablet_bootstrap.cc:654] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.780618 31363 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.783397 31363 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:19.785538 31184 ts_tablet_manager.cc:1434] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.002s
I20260709 12:23:19.786818 31184 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.804628 31184 tablet_bootstrap.cc:654] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.809412 31185 tablet_bootstrap.cc:492] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:19.809844 31185 ts_tablet_manager.cc:1403] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.205s	user 0.010s	sys 0.003s
I20260709 12:23:19.817050 31184 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.817492 31184 ts_tablet_manager.cc:1403] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.006s
I20260709 12:23:19.817423 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:19.820971 31184 raft_consensus.cc:359] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.821851 31184 raft_consensus.cc:385] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.822239 31184 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.823174 31186 ts_tablet_manager.cc:1403] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.409s	user 0.016s	sys 0.030s
I20260709 12:23:19.823174 31184 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.825938 31186 raft_consensus.cc:359] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.826508 31184 ts_tablet_manager.cc:1434] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:23:19.826637 31186 raft_consensus.cc:385] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.827010 31186 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.827636 31184 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.827641 31186 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.830005 31186 ts_tablet_manager.cc:1434] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:23:19.830850 31186 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:19.835662 31184 tablet_bootstrap.cc:654] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.838999 30991 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:19.839453 31186 tablet_bootstrap.cc:654] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.841274 31185 raft_consensus.cc:359] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.842037 31185 raft_consensus.cc:385] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.842315 31185 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.842983 31185 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.845216 31185 ts_tablet_manager.cc:1434] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 12:23:19.846151 31185 tablet_bootstrap.cc:492] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:19.846434 31184 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.846973 31184 ts_tablet_manager.cc:1403] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 12:23:19.849128 31363 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.849560 31363 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.851641 31363 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:19.853633 31185 tablet_bootstrap.cc:654] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.854530 31186 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:19.863711 31184 raft_consensus.cc:359] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.864549 31184 raft_consensus.cc:385] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:19.864880 31184 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:19.865614 31184 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:19.868721 31184 ts_tablet_manager.cc:1434] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:23:19.869813 31184 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:19.876956 31184 tablet_bootstrap.cc:654] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:19.888247 31184 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:19.888686 31184 ts_tablet_manager.cc:1403] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 12:23:19.972931 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:19.973201 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:19.974939 31064 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:19.981348 31449 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:19.981428 31106 leader_election.cc:343] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:19.984246 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:19.986091 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.987174 31192 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:19.991321 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.991456 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:19.992030 30992 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:19.992168 30988 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:20.003923 31265 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.004338 31265 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.004659 31265 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.006119 31186 ts_tablet_manager.cc:1403] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.175s	user 0.020s	sys 0.006s
I20260709 12:23:20.007335 31452 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:20.008757 31186 raft_consensus.cc:359] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.009384 31186 raft_consensus.cc:385] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.009645 31186 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.010489 31186 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.012765 31186 ts_tablet_manager.cc:1434] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:20.013588 31186 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:20.033404 31186 tablet_bootstrap.cc:654] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.051357 31184 raft_consensus.cc:359] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.052139 31184 raft_consensus.cc:385] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.052453 31184 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.053171 31184 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.039580 31265 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:20.055356 31109 leader_election.cc:343] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.065675 31184 ts_tablet_manager.cc:1434] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.177s	user 0.011s	sys 0.000s
I20260709 12:23:20.054580 31440 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:23:20.070966 31185 tablet_bootstrap.cc:492] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:20.071291 31185 ts_tablet_manager.cc:1403] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.225s	user 0.012s	sys 0.012s
I20260709 12:23:20.076726 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:20.067905 31276 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:23:20.089982 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:20.096786 31440 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.068500 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:20.094014 31185 raft_consensus.cc:359] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.074575 31265 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:20.099936 31185 raft_consensus.cc:385] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.101109 31185 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.101647 31185 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:20.102836 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48736.4us
I20260709 12:23:20.103407 31265 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.103574 31185 ts_tablet_manager.cc:1434] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:23:20.104597 31185 tablet_bootstrap.cc:492] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:20.106554 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:20.110913 31059 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.113049 31449 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.105496 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:20.113797 31449 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.118673 31443 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.119226 31443 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.120487 31443 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.212230 31192 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:23:20.212862 31192 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:20.215401 31186 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:20.215868 31186 ts_tablet_manager.cc:1403] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.202s	user 0.030s	sys 0.050s
I20260709 12:23:20.240478 31192 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.242301 31192 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:20.242754 31059 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:20.245265 31443 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:20.238700 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 133280us
W20260709 12:23:20.264580 31106 leader_election.cc:343] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.269946 31443 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:20.272722 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:20.272801 31449 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:20.287127 31184 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:20.287179 30990 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:20.272734 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:20.282367 31321 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.307106 31439 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:20.307868 31439 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:20.307123 31440 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:23:20.308040 31186 raft_consensus.cc:359] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.309042 31186 raft_consensus.cc:385] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.309406 31186 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.309660 31471 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.310165 31471 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.310302 31186 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.312700 31471 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:20.308159 31439 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:20.313802 31186 ts_tablet_manager.cc:1434] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.001s
I20260709 12:23:20.319223 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:20.304572 31321 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:20.330474 31185 tablet_bootstrap.cc:654] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.331055 31184 tablet_bootstrap.cc:654] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.318305 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:20.336344 31109 leader_election.cc:343] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.336799 31109 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:20.339493 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:20.321866 31439 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.331847 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:20.324311 31471 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:23:20.357919 31321 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.358894 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:20.359589 31139 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:20.360316 31449 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.360807 31449 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.355291 31265 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.342859 31226 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:20.362640 31067 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.373318 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:20.374701 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:20.375182 30992 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:20.375435 30955 leader_election.cc:343] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.375355 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:20.377054 30988 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.379253 31449 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:20.373356 31439 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:20.387090 30992 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:20.390087 31186 tablet_bootstrap.cc:492] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:20.392395 30885 catalog_manager.cc:5697] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:20.414898 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 50588.6us
I20260709 12:23:20.432863 31475 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.421904 31361 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.433369 31475 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.468580 31475 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:20.447710 31361 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.471581 30988 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:20.473894 31067 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:20.646910 31185 tablet_bootstrap.cc:492] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:20.647424 31185 ts_tablet_manager.cc:1403] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.543s	user 0.010s	sys 0.013s
I20260709 12:23:20.648412 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:23:20.652349 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 205923us
I20260709 12:23:20.659360 31265 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.663173 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:20.663203 31265 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:20.663362 31476 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.160s	user 0.006s	sys 0.005s
I20260709 12:23:20.666793 31186 tablet_bootstrap.cc:654] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.657878 31285 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.667228 31285 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.668704 31285 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.665059 31361 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:20.671998 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:20.672353 31109 leader_election.cc:343] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.653254 31226 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:23:20.686277 31265 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:20.688300 31106 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:20.691632 31185 raft_consensus.cc:359] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.692353 31185 raft_consensus.cc:385] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.692628 31185 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.687680 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:20.695318 31285 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.698611 31466 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.709827 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:20.709983 31466 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:20.710593 31135 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:20.714018 31185 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.711160 31466 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.724360 31185 ts_tablet_manager.cc:1434] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.002s
I20260709 12:23:20.725067 31185 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:20.729490 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:20.734925 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:20.735666 31276 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:20.736012 31276 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.736323 31276 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:20.723501 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50456.7us
I20260709 12:23:20.744009 31285 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:20.744493 31184 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:20.744936 31184 ts_tablet_manager.cc:1403] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.458s	user 0.006s	sys 0.026s
I20260709 12:23:20.737977 31275 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260709 12:23:20.748801 31033 leader_election.cc:343] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.753155 31186 tablet_bootstrap.cc:492] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:20.753700 31186 ts_tablet_manager.cc:1403] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.364s	user 0.005s	sys 0.009s
I20260709 12:23:20.750485 31361 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:23:20.769572 31184 raft_consensus.cc:359] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.769763 31186 raft_consensus.cc:359] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.770272 31184 raft_consensus.cc:385] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.770622 31184 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.770500 31186 raft_consensus.cc:385] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.771014 31186 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.771168 31184 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.773168 31186 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.777236 31184 ts_tablet_manager.cc:1434] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 12:23:20.778187 31186 ts_tablet_manager.cc:1434] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:23:20.779110 31186 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:20.777841 31276 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:20.757529 31109 leader_election.cc:343] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.797441 31276 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:20.799860 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:20.800516 31060 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.802131 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:20.802472 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:20.802667 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:20.803251 31144 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.803596 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:20.804324 31067 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 1: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:23:20.805979 31033 leader_election.cc:343] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:20.810606 31109 leader_election.cc:343] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.812016 31184 tablet_bootstrap.cc:492] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:20.817870 31186 tablet_bootstrap.cc:654] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.818424 31144 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:20.823011 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:20.824901 31275 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.827028 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:20.826045 31275 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:20.828326 31275 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.843792 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:20.844609 31137 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:20.845403 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:20.860036 31185 tablet_bootstrap.cc:654] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.868006 31466 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.868507 31466 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.871650 31466 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:23:20.872502 31106 leader_election.cc:343] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.872929 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:20.874253 31466 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.878818 31321 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:20.880731 31106 leader_election.cc:343] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.881150 31106 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:20.884042 31466 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.890558 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:23:20.892048 31489 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:23:20.895277 31060 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:20.895373 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:20.899994 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08001d1c00 after lost signal to thread 29848
I20260709 12:23:20.902383 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:23:20.905058 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08001d9e40 after lost signal to thread 30924
W20260709 12:23:20.906142 31165 debug-util.cc:398] Leaking SignalData structure 0x7b080025da40 after lost signal to thread 31016
W20260709 12:23:20.907090 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08002801c0 after lost signal to thread 31093
W20260709 12:23:20.908095 31165 debug-util.cc:398] Leaking SignalData structure 0x7b0800261c40 after lost signal to thread 31168
W20260709 12:23:20.907011 31109 leader_election.cc:343] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:20.914327 31106 leader_election.cc:343] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.893181 31466 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.916397 31137 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:20.914496 31226 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:20.926566 31226 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:20.915011 31106 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:20.927577 31226 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.928407 31321 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:20.942104 30938 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:20.943938 31184 tablet_bootstrap.cc:654] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:20.945225 30885 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 71) took 1034 ms. Trace:
W20260709 12:23:20.948865 31109 leader_election.cc:343] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.947098 30885 rpcz_store.cc:276] 0709 12:23:19.911143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:19.996033 (+ 84890us) service_pool.cc:224] Handling call
0709 12:23:20.945162 (+949129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:20.640112 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:20.641033 (+   921us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:20.641076 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:23:20.641097 (+    21us) tablet.cc:662] Decoding operations
0709 12:23:20.641525 (+   428us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:20.641616 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:20.641660 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:20.641878 (+   218us) tablet.cc:701] Row locks acquired
0709 12:23:20.641902 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:23:20.642134 (+   232us) write_op.cc:270] Start()
0709 12:23:20.651864 (+  9730us) write_op.cc:276] Timestamp: P: 1783599800642080 usec, L: 0
0709 12:23:20.657474 (+  5610us) op_driver.cc:348] REPLICATION: starting
0709 12:23:20.689045 (+ 31571us) log.cc:844] Serialized 537 byte log entry
0709 12:23:20.834805 (+145760us) op_driver.cc:464] REPLICATION: finished
0709 12:23:20.892634 (+ 57829us) write_op.cc:301] APPLY: starting
0709 12:23:20.892713 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:20.892819 (+   106us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:20.893456 (+   637us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:20.912208 (+ 18752us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:20.939988 (+ 27780us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:23:20.940261 (+   273us) write_op.cc:312] APPLY: finished
0709 12:23:20.940448 (+   187us) log.cc:844] Serialized 22 byte log entry
0709 12:23:20.940599 (+   151us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:20.940676 (+    77us) write_op.cc:454] Released schema lock
0709 12:23:20.940714 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:20.941049 (+   335us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:20.941084 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":386823,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":57604,"apply.run_cpu_time_us":1768,"apply.run_wall_time_us":48407,"num_ops":1,"prepare.queue_time_us":242818,"prepare.run_cpu_time_us":27043,"prepare.run_wall_time_us":82085,"raft.queue_time_us":68573,"raft.run_cpu_time_us":45824,"raft.run_wall_time_us":55448,"replication_time_us":177198,"thread_start_us":150973,"threads_started":4,"wal-append.queue_time_us":57850}]]}
I20260709 12:23:20.963419 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:20.963650 30887 catalog_manager.cc:5697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:20.970912 31186 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:20.971380 31186 ts_tablet_manager.cc:1403] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.192s	user 0.007s	sys 0.012s
I20260709 12:23:20.976202 31186 raft_consensus.cc:359] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:20.977412 31186 raft_consensus.cc:385] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:20.977644 31186 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:20.978639 31186 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:20.979815 31106 leader_election.cc:343] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:20.980274 31106 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:20.981515 31321 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:20.985802 31186 ts_tablet_manager.cc:1434] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:23:20.986716 31186 tablet_bootstrap.cc:492] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:20.992326 31361 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:20.993618 31276 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.994081 31276 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.000227 31276 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.004021 30886 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:21.005987 30886 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:21.014447 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:21.015396 31135 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:21.017757 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:21.017149 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:21.029189 31276 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.032655 31276 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.033000 31276 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.040481 31184 tablet_bootstrap.cc:492] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:21.041010 31184 ts_tablet_manager.cc:1403] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.229s	user 0.012s	sys 0.024s
I20260709 12:23:21.043581 31184 raft_consensus.cc:359] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.044225 31184 raft_consensus.cc:385] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.044488 31184 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.047514 31184 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:21.049486 31033 leader_election.cc:343] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.049741 31184 ts_tablet_manager.cc:1434] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:23:21.050755 31184 tablet_bootstrap.cc:492] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:21.052227 31276 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.056263 31185 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:21.056706 31185 ts_tablet_manager.cc:1403] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.332s	user 0.015s	sys 0.010s
I20260709 12:23:21.059881 31443 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:21.062644 30886 catalog_manager.cc:5697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.070415 31186 tablet_bootstrap.cc:654] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.078342 31185 raft_consensus.cc:359] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.079044 31185 raft_consensus.cc:385] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.079327 31185 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.079998 31185 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.082909 31185 ts_tablet_manager.cc:1434] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:23:21.083801 31185 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:21.085860 30887 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:21.093502 31184 tablet_bootstrap.cc:654] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.100284 31226 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.100757 31226 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.101825 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:23:21.103497 31033 leader_election.cc:343] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.104467 31226 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.104803 31276 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.106379 30885 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.107045 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:21.107649 31135 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.108630 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:21.107431 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:21.111589 31137 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
W20260709 12:23:21.112814 30955 leader_election.cc:343] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.113873 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:21.120467 31226 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.122210 31226 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.122581 31226 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.128585 31185 tablet_bootstrap.cc:654] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.130033 31186 tablet_bootstrap.cc:492] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:21.130522 31186 ts_tablet_manager.cc:1403] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.144s	user 0.015s	sys 0.011s
I20260709 12:23:21.143821 31135 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:21.145026 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:21.147608 31184 tablet_bootstrap.cc:492] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:21.148068 31184 ts_tablet_manager.cc:1403] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.098s	user 0.016s	sys 0.011s
I20260709 12:23:21.147814 31186 raft_consensus.cc:359] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.148471 31186 raft_consensus.cc:385] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.148742 31186 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.149214 31488 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.149389 31186 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.150691 31184 raft_consensus.cc:359] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.149672 31488 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.151343 31184 raft_consensus.cc:385] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.151765 31184 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.152315 31186 ts_tablet_manager.cc:1434] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:23:21.153054 31485 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.153532 31485 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:21.155774 31184 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.159081 31488 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:21.159757 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:21.160202 31226 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.160576 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:21.161880 31109 leader_election.cc:343] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:21.162552 31106 leader_election.cc:343] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.162940 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:21.154798 31485 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.163890 31321 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.166188 31226 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.167016 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:21.167582 31067 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.171885 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:21.172461 31135 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.174175 31184 ts_tablet_manager.cc:1434] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:23:21.174306 31186 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:21.174585 31067 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:21.175091 31184 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:21.179575 31135 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:21.181012 30955 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:21.181856 31226 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.182358 31226 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:21.183218 31226 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.191974 31186 tablet_bootstrap.cc:654] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.201093 31186 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:21.201752 31186 ts_tablet_manager.cc:1403] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.002s
I20260709 12:23:21.204296 31186 raft_consensus.cc:359] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.204886 31186 raft_consensus.cc:385] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.205137 31186 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.210886 31186 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.212905 31186 ts_tablet_manager.cc:1434] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 12:23:21.213014 30885 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.213795 31186 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:21.216681 31185 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:21.217099 31185 ts_tablet_manager.cc:1403] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.133s	user 0.027s	sys 0.009s
I20260709 12:23:21.219746 31185 raft_consensus.cc:359] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.220769 31186 tablet_bootstrap.cc:654] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.211287 31184 tablet_bootstrap.cc:654] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.220345 31185 raft_consensus.cc:385] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.235085 30887 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.235812 31185 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.236510 31185 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.240854 31185 ts_tablet_manager.cc:1434] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 12:23:21.241988 31185 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:21.248255 31185 tablet_bootstrap.cc:654] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.253333 31464 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.253839 31464 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.255857 31464 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.259912 31488 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.260064 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:21.260728 31063 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:21.262378 30955 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:21.263593 31226 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.263949 31226 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.264292 31226 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.265960 31184 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:21.266534 31184 ts_tablet_manager.cc:1403] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.001s
I20260709 12:23:21.266709 31186 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:21.267144 31186 ts_tablet_manager.cc:1403] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.006s
I20260709 12:23:21.269984 31186 raft_consensus.cc:359] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.260128 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:21.270648 31186 raft_consensus.cc:385] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.270828 31135 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:21.271000 31186 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.270402 31184 raft_consensus.cc:359] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.271431 31184 raft_consensus.cc:385] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.271777 31184 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.271665 31186 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.272658 31184 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.274125 31186 ts_tablet_manager.cc:1434] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:21.275043 31186 tablet_bootstrap.cc:492] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:21.274145 31184 ts_tablet_manager.cc:1434] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:21.276638 31226 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.279294 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:21.279839 31067 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.279919 31231 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.280936 31186 tablet_bootstrap.cc:654] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.288786 31184 tablet_bootstrap.cc:492] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:21.292392 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:21.297473 31137 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.303774 31137 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:21.304682 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:21.305416 31464 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.305936 31464 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:21.306797 31464 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.308287 31184 tablet_bootstrap.cc:654] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.295228 31067 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:21.295627 31226 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.295732 31186 tablet_bootstrap.cc:492] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:21.312664 31186 ts_tablet_manager.cc:1403] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.008s
I20260709 12:23:21.322273 30988 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.322734 30887 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.324734 31186 raft_consensus.cc:359] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.331007 31184 tablet_bootstrap.cc:492] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:21.334236 31186 raft_consensus.cc:385] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.334483 31184 ts_tablet_manager.cc:1403] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.000s
I20260709 12:23:21.334607 31186 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.335466 31186 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.336237 31231 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.337780 31186 ts_tablet_manager.cc:1434] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:23:21.338115 31184 raft_consensus.cc:359] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.338819 31184 raft_consensus.cc:385] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.339143 31184 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.339859 31184 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.341401 31186 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:21.347894 31186 tablet_bootstrap.cc:654] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.352603 31185 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:21.352825 31184 ts_tablet_manager.cc:1434] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:23:21.353025 31185 ts_tablet_manager.cc:1403] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.111s	user 0.012s	sys 0.000s
I20260709 12:23:21.357968 31285 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.364862 31488 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.364907 31185 raft_consensus.cc:359] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.371367 31185 raft_consensus.cc:385] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.371834 31185 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.372390 31184 tablet_bootstrap.cc:492] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:21.372489 31185 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.374972 31185 ts_tablet_manager.cc:1434] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:23:21.380064 31185 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:21.399541 31184 tablet_bootstrap.cc:654] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.405426 31185 tablet_bootstrap.cc:654] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.409813 31226 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.444162 31186 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:21.444692 31186 ts_tablet_manager.cc:1403] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.010s
I20260709 12:23:21.447297 31186 raft_consensus.cc:359] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.447939 31186 raft_consensus.cc:385] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.448194 31186 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.450328 31186 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.465407 31186 ts_tablet_manager.cc:1434] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:23:21.466995 31186 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:21.474722 31185 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:21.486524 31185 ts_tablet_manager.cc:1403] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.107s	user 0.029s	sys 0.025s
I20260709 12:23:21.490461 31226 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.491148 31226 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.493502 31226 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.500754 31184 tablet_bootstrap.cc:492] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:21.501207 31184 ts_tablet_manager.cc:1403] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.129s	user 0.014s	sys 0.035s
I20260709 12:23:21.500481 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:21.502267 31185 raft_consensus.cc:359] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.502995 31185 raft_consensus.cc:385] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.503254 31185 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.503466 31186 tablet_bootstrap.cc:654] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.503919 31185 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.504738 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:21.505329 31063 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:21.505961 31185 ts_tablet_manager.cc:1434] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.000s
I20260709 12:23:21.506863 31185 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:21.506844 30955 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:21.508374 31285 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.508754 31285 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.509088 31285 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.507145 31141 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:21.516160 31285 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.516274 31184 raft_consensus.cc:359] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.516987 31184 raft_consensus.cc:385] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.517278 31184 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.518942 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:21.519510 31063 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.522601 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:21.523141 31141 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.526562 31184 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.528496 31063 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:21.529451 31141 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:21.529757 30955 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:21.530563 31226 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.530619 31285 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.531291 31226 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:21.532137 31226 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.539366 31185 tablet_bootstrap.cc:654] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.542712 31184 ts_tablet_manager.cc:1434] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.003s
I20260709 12:23:21.543385 31184 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:21.552325 30887 catalog_manager.cc:5697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 0 to 1, leader changed from <none> to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:21.561094 31226 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.563136 31226 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.565618 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:21.575636 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:21.576452 31141 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 1: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:23:21.577698 31488 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.578378 31226 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.578255 31488 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.580986 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:21.581656 31186 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:21.582124 31186 ts_tablet_manager.cc:1403] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.115s	user 0.012s	sys 0.011s
W20260709 12:23:21.582466 31109 leader_election.cc:343] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.584002 31488 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:21.585036 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:21.584537 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:21.586253 31285 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:21.586711 31106 leader_election.cc:343] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.587054 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:21.588454 31464 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.588721 31186 raft_consensus.cc:359] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.589378 31186 raft_consensus.cc:385] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.589636 31186 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.590754 31488 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.591140 31186 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.594270 31186 ts_tablet_manager.cc:1434] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:23:21.595108 31285 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:21.596573 31186 tablet_bootstrap.cc:492] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:21.599197 31488 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.599653 31488 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.607278 31488 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:21.608999 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:21.609609 31064 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
I20260709 12:23:21.610632 31184 tablet_bootstrap.cc:654] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.610693 31485 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.612948 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:21.615093 31109 leader_election.cc:343] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.616590 31106 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:21.617672 31443 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.618053 31443 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.618407 31443 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.619220 31185 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:21.631165 31443 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.634388 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:23:21.635998 31109 leader_election.cc:343] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.636519 31226 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.636793 31185 ts_tablet_manager.cc:1403] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.130s	user 0.019s	sys 0.001s
I20260709 12:23:21.638486 31186 tablet_bootstrap.cc:654] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.641043 31185 raft_consensus.cc:359] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.641718 31185 raft_consensus.cc:385] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.641978 31185 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.648722 31185 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.659931 31185 ts_tablet_manager.cc:1434] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260709 12:23:21.666733 31185 tablet_bootstrap.cc:492] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:21.664522 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:21.663609 31443 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:21.674324 31063 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.690339 31063 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:21.693943 31106 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:21.694180 31184 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:21.694723 31184 ts_tablet_manager.cc:1403] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.151s	user 0.005s	sys 0.019s
I20260709 12:23:21.694947 31452 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:21.693110 31185 tablet_bootstrap.cc:654] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.697906 31452 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:21.697927 31184 raft_consensus.cc:359] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.703336 31184 raft_consensus.cc:385] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.703596 31184 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.704238 31184 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.706333 31184 ts_tablet_manager.cc:1434] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:23:21.707146 31184 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:21.707795 31452 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.790148 31186 tablet_bootstrap.cc:492] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:21.790877 31186 ts_tablet_manager.cc:1403] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.195s	user 0.012s	sys 0.011s
I20260709 12:23:21.782780 31231 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.813333 31186 raft_consensus.cc:359] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.814044 31186 raft_consensus.cc:385] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.814333 31186 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.814919 31186 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.816761 31186 ts_tablet_manager.cc:1434] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.001s
I20260709 12:23:21.817595 31186 tablet_bootstrap.cc:492] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:21.801616 31231 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.834587 31488 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.855536 31231 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:21.847159 31536 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:23:21.862748 31536 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.864765 31536 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:21.872535 31231 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.873360 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:21.873993 30984 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:21.875866 31185 tablet_bootstrap.cc:492] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:21.875873 31184 tablet_bootstrap.cc:654] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.876310 31185 ts_tablet_manager.cc:1403] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.210s	user 0.024s	sys 0.031s
I20260709 12:23:21.869666 31537 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.867791 31488 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.887221 31471 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:21.885461 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:21.884608 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:21.892535 31140 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:21.892710 31185 raft_consensus.cc:359] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.893386 31185 raft_consensus.cc:385] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:21.893743 31185 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:21.895015 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:21.895627 31067 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:21.894374 31185 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.893437 31488 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:21.907526 30885 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:21.891642 31485 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:21.910429 31485 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.910790 31485 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:21.913439 31106 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:21.926726 31185 ts_tablet_manager.cc:1434] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.050s	user 0.011s	sys 0.003s
I20260709 12:23:21.929987 31185 tablet_bootstrap.cc:492] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:21.930073 31186 tablet_bootstrap.cc:654] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:21.885654 31537 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:21.903491 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:21.940027 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:21.937620 31452 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:21.939308 31543 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.954262 31452 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:23:21.962812 30984 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:23:21.975251 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:21.986851 31537 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:21.970139 31452 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:22.040030 31106 leader_election.cc:343] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:21.959808 31543 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.041458 31285 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.043493 31285 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.045480 31285 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.047335 31184 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:22.063313 31184 ts_tablet_manager.cc:1403] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.356s	user 0.016s	sys 0.019s
W20260709 12:23:22.072077 31490 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:23:22.090755 31485 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.092760 31485 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.092695 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:22.094739 31452 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.096640 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:22.098212 31452 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:22.100776 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:22.101564 30990 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:22.102507 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:22.103036 31144 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.106564 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.107146 31136 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:22.110055 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:22.110626 30992 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:22.115463 31106 leader_election.cc:343] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.120673 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:23:22.122326 31109 leader_election.cc:343] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.122833 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.096668 31543 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.126546 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:22.130925 31144 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
W20260709 12:23:22.145174 31109 leader_election.cc:343] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.133252 31543 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.128470 31545 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.146023 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.153754 30992 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:22.155799 31536 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.156103 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:22.160048 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.160661 31144 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 0.
I20260709 12:23:22.154438 31543 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.166070 31543 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.172679 31060 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:22.173816 31106 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:22.174649 31542 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:22.160842 31551 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:22.175091 31542 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:22.175583 31551 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:22.175994 31542 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.176708 31551 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.182343 31452 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.183485 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:22.184394 30992 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:23:22.195065 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:22.195708 31226 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.196027 31226 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.196323 31226 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.205369 31185 tablet_bootstrap.cc:654] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.189368 31452 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.210150 31184 raft_consensus.cc:359] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.213088 31184 raft_consensus.cc:385] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.213382 31184 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.216190 31452 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:22.219812 31186 tablet_bootstrap.cc:492] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:22.220268 31186 ts_tablet_manager.cc:1403] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.403s	user 0.011s	sys 0.011s
I20260709 12:23:22.226760 31226 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:22.233289 30938 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:22.219832 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:22.234071 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:22.235471 31109 leader_election.cc:343] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.235652 31464 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:22.236670 31106 leader_election.cc:343] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.237102 31106 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.241969 31226 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.242857 31186 raft_consensus.cc:359] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.243368 31186 raft_consensus.cc:385] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.243584 31186 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.244129 31186 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.239903 31536 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.247681 31543 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.247961 31184 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.249621 31543 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.256609 31186 ts_tablet_manager.cc:1434] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.002s
I20260709 12:23:22.257386 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:22.257993 31139 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.269599 31139 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:22.273850 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:22.274607 31142 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 1: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:23:22.274690 31186 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:22.278858 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:22.278659 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:22.280342 30989 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 1: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:23:22.280628 31063 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 1: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:23:22.274048 31184 ts_tablet_manager.cc:1434] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.210s	user 0.008s	sys 0.001s
I20260709 12:23:22.284855 31184 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:22.286684 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.287875 31550 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:22.288332 31550 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:22.289196 31550 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.291493 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:22.295890 31485 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.296427 31485 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.298439 31285 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.301764 31485 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.306072 31464 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.307224 30887 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.311303 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:22.313021 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.313668 31185 tablet_bootstrap.cc:492] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:22.314328 31185 ts_tablet_manager.cc:1403] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.387s	user 0.013s	sys 0.005s
W20260709 12:23:22.320755 31033 leader_election.cc:343] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.313627 31141 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:22.330664 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:22.333153 31226 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.337179 31184 tablet_bootstrap.cc:654] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.332901 31185 raft_consensus.cc:359] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.349627 31186 tablet_bootstrap.cc:654] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.346802 31185 raft_consensus.cc:385] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.350328 31185 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.350469 30885 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.353916 30885 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:22.361521 31109 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:22.368288 31185 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.371392 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.392976 31185 ts_tablet_manager.cc:1434] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.000s
I20260709 12:23:22.393873 31185 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:22.400060 31184 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:22.402613 31184 ts_tablet_manager.cc:1403] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.118s	user 0.012s	sys 0.001s
I20260709 12:23:22.377660 31485 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:23:22.418949 31562 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:23:22.411020 31485 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.425254 31485 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.438134 31184 raft_consensus.cc:359] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.439889 31184 raft_consensus.cc:385] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.440158 31184 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.441344 31184 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.443516 31184 ts_tablet_manager.cc:1434] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.000s
I20260709 12:23:22.444375 31184 tablet_bootstrap.cc:492] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:22.454192 31185 tablet_bootstrap.cc:654] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.456410 31186 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:22.456880 31186 ts_tablet_manager.cc:1403] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.182s	user 0.016s	sys 0.010s
I20260709 12:23:22.456593 31542 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.461473 31485 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.467365 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:22.467861 30992 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.468000 31545 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.468474 31545 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.473964 30992 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:22.476794 31186 raft_consensus.cc:359] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.477589 31186 raft_consensus.cc:385] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.477895 31186 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.478783 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:22.479393 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:22.479974 31140 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.482304 31485 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.482383 31543 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:22.482601 31186 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.482967 31543 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:22.483804 31543 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.494995 31186 ts_tablet_manager.cc:1434] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 12:23:22.495832 31186 tablet_bootstrap.cc:492] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:22.497053 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:22.498373 31109 leader_election.cc:343] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.498618 31545 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:22.500061 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:23:22.501427 31106 leader_election.cc:343] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.501938 31106 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.503180 31542 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.504622 31186 tablet_bootstrap.cc:654] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.505659 31184 tablet_bootstrap.cc:654] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.505142 31140 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:22.520118 30885 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.530346 31184 tablet_bootstrap.cc:492] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:22.530540 31186 tablet_bootstrap.cc:492] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:22.530812 31184 ts_tablet_manager.cc:1403] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.087s	user 0.009s	sys 0.002s
I20260709 12:23:22.530922 31186 ts_tablet_manager.cc:1403] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.035s	user 0.005s	sys 0.008s
I20260709 12:23:22.532738 31185 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:22.533053 31185 ts_tablet_manager.cc:1403] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.139s	user 0.012s	sys 0.010s
I20260709 12:23:22.541332 31186 raft_consensus.cc:359] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.542068 31186 raft_consensus.cc:385] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.542376 31186 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.543012 31186 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.545276 31186 ts_tablet_manager.cc:1434] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260709 12:23:22.546952 31186 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:22.553781 31186 tablet_bootstrap.cc:654] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.555732 31184 raft_consensus.cc:359] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.556236 31184 raft_consensus.cc:385] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.556449 31184 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.557853 31184 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.561211 31184 ts_tablet_manager.cc:1434] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 12:23:22.562494 31184 tablet_bootstrap.cc:492] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:22.563578 31185 raft_consensus.cc:359] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.564432 31185 raft_consensus.cc:385] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.564760 31185 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.569399 31185 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.576598 31543 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.577001 31543 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.578615 31543 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.579356 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:22.580767 31033 leader_election.cc:343] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.581812 31185 ts_tablet_manager.cc:1434] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260709 12:23:22.582573 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.583031 31185 tablet_bootstrap.cc:492] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:22.583139 31141 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:22.608512 31464 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.608793 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.609062 31464 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.611886 30992 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:22.612181 31184 tablet_bootstrap.cc:654] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.612125 31464 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.612032 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:22.612964 31063 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:22.614102 31186 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:22.614634 31186 ts_tablet_manager.cc:1403] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.069s	user 0.043s	sys 0.015s
I20260709 12:23:22.614691 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:22.615032 31537 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.615440 31464 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:22.615890 31464 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.615509 31537 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.616230 31464 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.617118 31540 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.617475 31540 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.617707 31537 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:22.617786 31540 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.612032 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.619382 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:22.619978 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:22.620564 31106 leader_election.cc:343] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:22.621301 31109 leader_election.cc:343] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.621666 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.622609 31542 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.623787 31550 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.624202 31550 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.624338 31540 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.628338 31540 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.627759 31186 raft_consensus.cc:359] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.629082 31186 raft_consensus.cc:385] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.630134 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:22.632679 31186 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.633293 31186 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.633749 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:22.634281 31141 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.634795 31186 ts_tablet_manager.cc:1434] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 12:23:22.635638 31186 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:22.637746 31185 tablet_bootstrap.cc:654] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.631515 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:22.641642 31537 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.643255 31186 tablet_bootstrap.cc:654] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:22.644459 31033 leader_election.cc:343] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:22.650697 31033 leader_election.cc:343] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.652673 31141 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:22.653893 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.653770 31464 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.655800 31184 tablet_bootstrap.cc:492] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:22.656224 31184 ts_tablet_manager.cc:1403] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.094s	user 0.010s	sys 0.000s
I20260709 12:23:22.656268 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:22.657051 31063 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.658668 31540 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:22.659159 31540 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:22.659929 31540 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.662096 31184 raft_consensus.cc:359] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.662810 30989 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:23:22.663826 31063 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:23:22.664880 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:22.665593 31471 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:22.666043 31471 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:22.666818 31471 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.679263 31542 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.680723 31550 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.680267 31545 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:22.681916 31545 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:22.662820 31184 raft_consensus.cc:385] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.682600 31184 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.683238 31184 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.683981 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.684734 31140 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:22.685883 31185 tablet_bootstrap.cc:492] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:22.686338 31464 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.686375 31185 ts_tablet_manager.cc:1403] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.104s	user 0.009s	sys 0.004s
I20260709 12:23:22.687984 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:22.685900 31545 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.690168 31545 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.692538 31184 ts_tablet_manager.cc:1434] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:23:22.696794 31184 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:22.700870 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.707129 31549 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.707466 31549 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.707794 31549 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.710916 31485 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:22.731206 31185 raft_consensus.cc:359] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.731731 31185 raft_consensus.cc:385] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.731956 31185 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.740131 31549 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.742703 31185 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.744930 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:22.743185 31549 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:23:22.749022 31033 leader_election.cc:343] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.749763 31186 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:22.750190 31186 ts_tablet_manager.cc:1403] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.115s	user 0.012s	sys 0.010s
I20260709 12:23:22.750279 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:22.754257 31140 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.761600 31139 raft_consensus.cc:2410] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:23:22.765741 30885 catalog_manager.cc:5697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.765172 30886 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 2 to 3, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.768013 31485 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.770779 31185 ts_tablet_manager.cc:1434] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.000s
I20260709 12:23:22.771667 31185 tablet_bootstrap.cc:492] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:22.773419 31140 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:22.774590 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:22.775283 31485 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:22.776517 31485 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:22.780889 31485 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.782021 31184 tablet_bootstrap.cc:654] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.781922 31551 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.786679 31186 raft_consensus.cc:359] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.791451 31186 raft_consensus.cc:385] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.791702 31186 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.799505 31186 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.804420 31186 ts_tablet_manager.cc:1434] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.001s
I20260709 12:23:22.804957 31551 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.809458 31551 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.804803 31464 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.810631 31464 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.812019 30992 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.812588 31464 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.814271 31539 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.814121 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:22.819350 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.820835 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:22.822418 31551 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.814808 31539 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.825261 31539 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.827948 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.828771 31142 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:22.829934 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:22.830709 31141 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:22.830966 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:22.833377 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:22.840835 31186 tablet_bootstrap.cc:492] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:22.837961 31549 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.837205 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:22.845329 31549 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.845702 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:22.844529 31464 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:22.845729 31549 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:22.847074 31033 leader_election.cc:343] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:22.847893 31033 leader_election.cc:343] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.834789 31539 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:22.850481 30886 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.851432 31539 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.858654 31539 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.862187 31485 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:23:22.884342 31184 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:22.884771 31184 ts_tablet_manager.cc:1403] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.188s	user 0.016s	sys 0.003s
I20260709 12:23:22.889420 31186 tablet_bootstrap.cc:654] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.891048 31185 tablet_bootstrap.cc:654] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:22.896353 31184 raft_consensus.cc:359] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.896986 31184 raft_consensus.cc:385] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:22.897239 31184 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:22.897812 31184 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.901397 31539 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.903868 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:23:22.906579 31033 leader_election.cc:343] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:22.907400 31551 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:22.960052 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:22.956040 31539 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:22.971587 31139 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:22.983170 31549 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:22.986174 31485 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:22.998813 31549 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.000379 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:23:23.003993 31033 leader_election.cc:343] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.002283 31184 ts_tablet_manager.cc:1434] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.117s	user 0.004s	sys 0.005s
I20260709 12:23:23.000999 31139 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:23.005966 31185 tablet_bootstrap.cc:492] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:23.008026 31186 tablet_bootstrap.cc:492] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:23.015295 31184 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Bootstrap starting.
I20260709 12:23:23.015416 31186 ts_tablet_manager.cc:1403] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.175s	user 0.018s	sys 0.031s
I20260709 12:23:23.016239 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:23.017230 31586 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.017822 31586 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:23.018152 31186 raft_consensus.cc:359] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.018833 31186 raft_consensus.cc:385] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.019126 31186 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.018692 31586 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.019737 31186 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.015384 31185 ts_tablet_manager.cc:1403] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.244s	user 0.010s	sys 0.012s
I20260709 12:23:23.013633 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:23.020993 31142 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.022255 31186 ts_tablet_manager.cc:1434] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:23.023690 31186 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:23.032109 31550 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.032585 31550 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.043047 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:23.045117 31033 leader_election.cc:343] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.040107 31185 raft_consensus.cc:359] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.046444 31185 raft_consensus.cc:385] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.046940 31185 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.052129 31184 tablet_bootstrap.cc:654] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.057150 31550 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.058593 31185 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.096221 31185 ts_tablet_manager.cc:1434] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.003s
I20260709 12:23:23.096700 31142 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:23.098501 31185 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:23.101643 31184 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: No bootstrap required, opened a new log
I20260709 12:23:23.104867 31184 ts_tablet_manager.cc:1403] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Time spent bootstrapping tablet: real 0.090s	user 0.014s	sys 0.002s
I20260709 12:23:23.102075 30985 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.104142 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:23.109498 31139 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:23.112725 30886 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.114364 31186 tablet_bootstrap.cc:654] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.138872 31184 raft_consensus.cc:359] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.139914 31184 raft_consensus.cc:385] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.140282 31184 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.140902 31184 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.143987 31226 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.144979 31226 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.146239 31185 tablet_bootstrap.cc:654] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.147330 31226 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.148619 31140 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:23:23.149060 31140 pending_rounds.cc:85] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 1
I20260709 12:23:23.149354 31140 pending_rounds.cc:107] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:23:23.151966 31184 ts_tablet_manager.cc:1434] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.000s
I20260709 12:23:23.149663 31140 raft_consensus.cc:2889] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:23.152258 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:23.153267 31593 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.157254 31593 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.157374 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:23.160012 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:23.163666 31539 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.166162 31033 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:23.163694 31546 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.174142 31550 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.174441 31546 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.179417 31550 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:23.179886 31546 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:23.164834 31139 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:23.180241 31550 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.181977 31549 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.182394 31549 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.182700 31549 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.183194 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:23.183463 31186 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:23.183887 31186 ts_tablet_manager.cc:1403] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.160s	user 0.008s	sys 0.015s
I20260709 12:23:23.184592 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:23.186555 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:23.184547 31106 leader_election.cc:343] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.187311 31226 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:23.187734 31226 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.188006 31226 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.201370 31185 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:23.201905 31185 ts_tablet_manager.cc:1403] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.104s	user 0.012s	sys 0.019s
I20260709 12:23:23.198429 31595 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:23:23.204041 31593 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:23.205472 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:23:23.206566 31109 leader_election.cc:343] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.206964 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:23:23.207890 31109 leader_election.cc:343] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.205876 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:23.209570 31593 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.210403 31185 raft_consensus.cc:359] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:23.215517 31106 leader_election.cc:343] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.215811 31185 raft_consensus.cc:385] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.219744 31549 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.216099 31185 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.222122 31549 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.216027 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:23.223757 31185 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.228575 31185 ts_tablet_manager.cc:1434] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 12:23:23.228978 31542 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.229022 31060 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:23.234015 31464 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:23.239473 31185 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:23.245396 31185 tablet_bootstrap.cc:654] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.213464 31186 raft_consensus.cc:359] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.248025 31186 raft_consensus.cc:385] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.248328 31186 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.247773 30887 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.249017 31186 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.250394 31226 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.251238 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:23.251238 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:23.251839 31142 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.251454 30992 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:23:23.253808 31186 ts_tablet_manager.cc:1434] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.070s	user 0.004s	sys 0.003s
I20260709 12:23:23.254695 31186 tablet_bootstrap.cc:492] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:23.256038 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:23.256620 31140 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.257257 31542 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.257758 31542 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.260084 31542 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:23.263692 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:23.264292 31063 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 0.
W20260709 12:23:23.266812 31033 leader_election.cc:343] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.267287 30989 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.267836 31551 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.270292 31551 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.270624 31551 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:23.274351 31142 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:23.276237 31106 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:23.277041 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:23.279074 31537 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.271485 31551 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:23.281042 31109 leader_election.cc:343] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:23.282796 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.283236 31033 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:23.281044 31537 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.302680 31537 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.302838 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:23.302299 31586 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.252200 31226 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.348955 31186 tablet_bootstrap.cc:654] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.352958 31551 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.353843 31140 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:23.393610 31537 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.396135 31537 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:23.398659 31186 tablet_bootstrap.cc:492] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:23.399124 31186 ts_tablet_manager.cc:1403] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.145s	user 0.007s	sys 0.010s
I20260709 12:23:23.401743 31586 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:23.401687 31186 raft_consensus.cc:359] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.402347 31186 raft_consensus.cc:385] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.402607 31186 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.403070 31485 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.403205 31186 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.403540 31485 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.405388 31226 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.405573 31485 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.405894 31226 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.407692 31226 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.402683 31586 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.408882 31062 raft_consensus.cc:2410] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:23:23.409263 31186 ts_tablet_manager.cc:1434] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:23:23.412894 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:23.416478 31186 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Bootstrap starting.
I20260709 12:23:23.426963 31185 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:23.427410 31185 ts_tablet_manager.cc:1403] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.194s	user 0.018s	sys 0.012s
I20260709 12:23:23.430397 31185 raft_consensus.cc:359] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.431010 31185 raft_consensus.cc:385] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.431270 31185 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.431881 31185 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:23:23.434593 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 83645.5us
I20260709 12:23:23.437467 31186 tablet_bootstrap.cc:654] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:23.439141 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 154185us
I20260709 12:23:23.440999 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:23.445106 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:23.445838 31144 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 1: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:23:23.450524 31600 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:23.451097 31600 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:23.451901 31600 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.452780 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:23.453365 31063 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.450444 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:23.457607 31543 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.472196 30886 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.474243 31063 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:23:23.477320 31106 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:23.480037 31542 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.480511 31542 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:23.481279 31542 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.483976 31595 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.484449 31595 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.487515 31595 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.488245 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:23.489078 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:23.494638 31595 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.510681 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:23:23.512396 31033 leader_election.cc:343] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:23.523458 31109 leader_election.cc:343] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.540416 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:23.541522 31141 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:23.543592 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:23.540499 30886 catalog_manager.cc:5697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.558506 31141 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:23.562255 31185 ts_tablet_manager.cc:1434] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.135s	user 0.007s	sys 0.000s
I20260709 12:23:23.563844 31185 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:23.572702 31595 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.576354 31550 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.578648 31186 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: No bootstrap required, opened a new log
I20260709 12:23:23.579147 31186 ts_tablet_manager.cc:1403] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent bootstrapping tablet: real 0.165s	user 0.038s	sys 0.007s
I20260709 12:23:23.585253 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:23.588487 31550 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.588840 31550 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.589217 31550 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.591957 31186 raft_consensus.cc:359] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.592872 31186 raft_consensus.cc:385] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.593230 31186 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Initialized, Role: FOLLOWER
W20260709 12:23:23.596832 30938 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:23.601442 31186 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.604993 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:23.605706 31607 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.611814 31550 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.613796 31550 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.609889 31607 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.623096 30984 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.628410 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:23.629004 31135 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.629562 31186 ts_tablet_manager.cc:1434] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.002s
I20260709 12:23:23.633379 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:23.637696 31607 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:23:23.640589 31033 leader_election.cc:343] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.640666 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:23.621595 31600 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:23.649791 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:23.655392 30887 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.650650 31540 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.651396 31549 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.673389 31136 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 1: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:23:23.676785 31600 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:23:23.677304 31600 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.667246 31540 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.681581 30958 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:23.682245 31226 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:23.697101 31185 tablet_bootstrap.cc:654] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:23.699627 31135 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:23.700117 31540 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.712641 31600 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.713960 30984 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:23.721159 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 60670.3us
I20260709 12:23:23.725993 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:23.730372 31551 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:23.730837 31551 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:23.736109 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:23.736781 31136 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:23.736042 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:23.743877 31540 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.731626 31551 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.752840 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:23.754480 31549 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:23.754841 31549 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:23.755542 31549 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.756747 31540 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.757200 31576 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:23.757563 31576 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:23.757910 31576 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.767639 31616 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260709 12:23:23.779660 31033 leader_election.cc:343] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:23.783205 31185 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:23.783972 31185 ts_tablet_manager.cc:1403] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.220s	user 0.028s	sys 0.038s
I20260709 12:23:23.782711 31485 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.805505 31549 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:23.820458 31185 raft_consensus.cc:359] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.824951 31185 raft_consensus.cc:385] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:23.825307 31185 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:23.826797 31185 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.828702 31576 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.831319 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:23.828917 31185 ts_tablet_manager.cc:1434] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.000s
I20260709 12:23:23.833846 31600 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.836040 31067 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:23:23.836328 31067 pending_rounds.cc:85] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 0
I20260709 12:23:23.836520 31067 pending_rounds.cc:107] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:23:23.836728 31067 raft_consensus.cc:2889] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:23.839744 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:23.841898 31185 tablet_bootstrap.cc:492] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:23.842567 30989 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.845118 31607 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:23.873095 31576 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:23.957413 30885 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:23.969206 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:23.969805 31144 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:23.873406 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:23.936031 31586 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.990073 31586 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:23.992390 31586 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:24.001758 31144 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:23.972095 31540 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 12:23:23.972886 31062 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:24.010938 31185 tablet_bootstrap.cc:654] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.029652 31144 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:24.032749 31607 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:24.032924 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:24.033912 31144 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 0.
I20260709 12:23:24.026988 31471 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.051381 31471 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.056190 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:24.057490 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:24.063602 31471 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:24.064253 31595 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:24.064579 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:24.064889 31595 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:24.065101 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:24.065243 31139 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:23:24.066004 31595 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.068701 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:24.070852 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:24.080984 31615 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:24.081401 31615 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.081698 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:23:24.082747 31033 leader_election.cc:343] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:24.083755 31543 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:24.084182 31543 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:24.088740 31543 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.090444 31576 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:24.090828 31576 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:24.091169 31576 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:24.092037 31033 leader_election.cc:343] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:24.081718 31615 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:24.097950 31616 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:23:24.092216 31542 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
W20260709 12:23:24.120458 31109 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:24.158043 31615 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.162616 30989 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:23:24.163331 31615 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:24.166507 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:24.168141 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:24.170393 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:24.171561 31471 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:24.180974 31576 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.183143 31576 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:24.187988 30887 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:24.192318 31185 tablet_bootstrap.cc:492] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:24.192888 31185 ts_tablet_manager.cc:1403] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.354s	user 0.028s	sys 0.018s
W20260709 12:23:24.215268 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 60141.5us
I20260709 12:23:24.217036 31634 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:24.217878 31485 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:24.218286 31485 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:24.219105 31485 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.220263 31485 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:24.222855 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:24.229765 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:24.230374 31136 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.235265 31185 raft_consensus.cc:359] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.236192 31185 raft_consensus.cc:385] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.236445 31185 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.237134 31185 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.247889 31185 ts_tablet_manager.cc:1434] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.054s	user 0.003s	sys 0.003s
I20260709 12:23:24.248762 31185 tablet_bootstrap.cc:492] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
W20260709 12:23:24.252276 31033 leader_election.cc:343] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:24.252946 31033 consensus_peers.cc:597] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:24.270805 30886 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:24.278630 31136 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:23:24.280411 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:24.281092 31634 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:24.281539 31634 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:24.287012 31634 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.293660 31185 tablet_bootstrap.cc:654] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.310045 31185 tablet_bootstrap.cc:492] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:24.310516 31185 ts_tablet_manager.cc:1403] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.000s
I20260709 12:23:24.313123 31185 raft_consensus.cc:359] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.313783 31185 raft_consensus.cc:385] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.314078 31185 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.313791 30887 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:24.314904 31185 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.327639 31185 ts_tablet_manager.cc:1434] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:23:24.333127 31185 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:24.338680 31185 tablet_bootstrap.cc:654] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.343959 31586 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:24.369405 31185 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:24.369932 31185 ts_tablet_manager.cc:1403] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.022s
I20260709 12:23:24.400002 31185 raft_consensus.cc:359] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.400753 31185 raft_consensus.cc:385] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.401073 31185 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.404853 31185 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.407141 31185 ts_tablet_manager.cc:1434] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:23:24.408171 31185 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Bootstrap starting.
I20260709 12:23:24.414687 31615 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.415196 31615 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.416177 31185 tablet_bootstrap.cc:654] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:24.417249 31615 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:24.418807 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:24.419576 31144 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 1: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:23:24.423557 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:24.425706 30955 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:24.427461 31615 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.427946 31185 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: No bootstrap required, opened a new log
I20260709 12:23:24.428360 31185 ts_tablet_manager.cc:1403] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.004s
I20260709 12:23:24.431301 31185 raft_consensus.cc:359] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.432024 31185 raft_consensus.cc:385] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:24.432231 31595 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.432312 31185 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Initialized, Role: FOLLOWER
I20260709 12:23:24.432775 31595 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.433153 31185 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.434810 31595 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:24.435478 31185 ts_tablet_manager.cc:1434] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:24.436767 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:24.437444 31144 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 1: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:23:24.437311 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:24.439982 31615 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:24.442615 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:24.443506 31615 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.449800 31615 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:24.481539 31615 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.482086 31615 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.485292 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:24.486274 31615 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:24.487342 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:24.488539 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:24.489251 31615 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:24.543609 31464 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.545083 31464 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:24.553467 31464 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:24.558039 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:24.565442 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:24.576344 30985 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.577450 31543 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:24.604255 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:24.604557 31064 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.618590 31607 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:24.714592 31471 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:24.716091 31140 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:24.724509 31140 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:24.727279 31471 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:24.876477 30987 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:24.880522 31576 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:24.957787 31646 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:24.970537 31646 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:23:25.181967 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:25.208961 31651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:25.209021 31654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:25.209409 31652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:25.210829 29845 server_base.cc:1061] running on GCE node
I20260709 12:23:25.212421 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:25.212646 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:25.212833 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783599805212817 us; error 0 us; skew 500 ppm
I20260709 12:23:25.213409 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:25.216894 29845 webserver.cc:533] Webserver started at http://127.29.37.68:37881/ using document root <none> and password file <none>
I20260709 12:23:25.217404 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:25.217617 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:25.217941 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:25.219385 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-3-root/instance:
uuid: "a517aadb0ae04b89a13779d7898dfa0c"
format_stamp: "Formatted at 2026-07-09 12:23:25 on dist-test-slave-g1m6"
I20260709 12:23:25.224510 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:23:25.228243 31660 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:25.230593 29845 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:23:25.230921 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-3-root
uuid: "a517aadb0ae04b89a13779d7898dfa0c"
format_stamp: "Formatted at 2026-07-09 12:23:25 on dist-test-slave-g1m6"
I20260709 12:23:25.231503 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:25.264063 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:25.265605 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:25.267256 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:25.270592 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:25.270918 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.001s
I20260709 12:23:25.272550 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:25.272863 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:25.420528 31722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.68:35417 every 8 connection(s)
I20260709 12:23:25.421357 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.68:35417
I20260709 12:23:25.435597 29845 auto_leader_rebalancer-test.cc:439] add a tserver: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:25.457159 31723 heartbeater.cc:344] Connected to a master server at 127.29.37.126:44053
I20260709 12:23:25.457557 31723 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:25.458299 31723 heartbeater.cc:507] Master 127.29.37.126:44053 requested a full tablet report, sending...
I20260709 12:23:25.460408 30885 ts_manager.cc:194] Registered new tserver with Master: a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:23:25.462036 30885 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59312
I20260709 12:23:25.629240 31542 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:23:25.629930 31542 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:25.633177 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:23:25.634279 30989 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:23:25.636550 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:25.637202 31067 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:25.638106 31542 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:25.638573 31106 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:25.639377 31542 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:25.639820 31542 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:23:25.640214 31542 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:25.648293 31542 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:25.650852 31542 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:25.651537 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:25.652220 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:25.652968 30987 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:23:25.659531 31064 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:25.660846 31106 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:25.661549 31542 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:25.662138 31542 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:25.663285 31542 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:25.676419 30885 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:26.055384 31613 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.055944 31613 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:26.058943 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:23:26.058126 31613 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:26.059595 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:23:26.068666 31064 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:23:26.069941 30955 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:26.078938 31733 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:26.121034 31135 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:26.127447 31064 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:26.130438 30987 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:26.130486 31542 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:26.131954 31592 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:26.146100 31109 consensus_peers.cc:597] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9bb92178abcc4ef385f771bd04448a5b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:26.159837 31067 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.166687 31542 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.172551 30987 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.180967 31135 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:26.185740 30885 catalog_manager.cc:5697] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:26.201366 31068 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.207309 31536 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:26.219985 30985 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.222817 31064 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:26.226727 31536 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:26.238356 31542 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:26.247766 30989 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:26.258376 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 563f5eec53c14273959c9883d2423cc9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:26.264832 31542 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:26.292874 31592 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.303061 31064 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:23:26.335353 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58186.3us
I20260709 12:23:26.341720 30985 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.418499 30886 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:26.441166 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59591.8us
I20260709 12:23:26.465411 31723 heartbeater.cc:499] Master 127.29.37.126:44053 was elected leader, sending a full tablet report...
I20260709 12:23:26.466181 30985 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:26.485984 31062 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:26.487533 31613 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:26.492925 31135 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:26.524055 31613 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:23:26.544976 30958 consensus_peers.cc:597] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e3de96a6f20142c0a00ff9ee12bcc86c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:26.557420 31734 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.566639 31140 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.573436 31062 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.586483 30885 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.586647 30985 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
W20260709 12:23:26.616148 30958 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8ba55d3e2574f7a890f7a956ba4e74f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:26.616324 31062 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:26.617058 31140 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:26.618490 31734 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:26.619355 31735 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:26.628662 31734 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.632802 31062 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.648558 30885 catalog_manager.cc:5697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:26.656276 31140 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:26.705741 31768 ts_tablet_manager.cc:933] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:26.716377 31768 tablet_copy_client.cc:323] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:26.749974 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 9bb92178abcc4ef385f771bd04448a5b from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:26.750619 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 9bb92178abcc4ef385f771bd04448a5b from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-9bb92178abcc4ef385f771bd04448a5b
I20260709 12:23:26.760771 31154 tablet_copy_source_session.cc:215] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:26.765846 31768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb92178abcc4ef385f771bd04448a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:26.788944 31768 tablet_copy_client.cc:806] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:26.789732 31768 tablet_copy_client.cc:670] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:26.794721 31768 tablet_copy_client.cc:538] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:26.804400 31768 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:26.841670 31768 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:26.847757 31768 tablet_bootstrap.cc:492] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:26.849514 31768 ts_tablet_manager.cc:1403] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.045s	user 0.024s	sys 0.008s
I20260709 12:23:26.855206 31768 raft_consensus.cc:359] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:26.851326 31773 ts_tablet_manager.cc:933] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:26.858080 31768 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:26.858878 31768 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:26.877552 31773 tablet_copy_client.cc:323] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:26.880995 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 563f5eec53c14273959c9883d2423cc9 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:26.881543 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 563f5eec53c14273959c9883d2423cc9 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-563f5eec53c14273959c9883d2423cc9
I20260709 12:23:26.886758 31768 ts_tablet_manager.cc:1434] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.007s
I20260709 12:23:26.889704 31153 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-9bb92178abcc4ef385f771bd04448a5b received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:26.890230 31153 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-9bb92178abcc4ef385f771bd04448a5b on tablet 9bb92178abcc4ef385f771bd04448a5b with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:26.890743 31776 ts_tablet_manager.cc:933] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:26.891800 31154 tablet_copy_source_session.cc:215] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:26.896890 31773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 563f5eec53c14273959c9883d2423cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:26.900897 31776 tablet_copy_client.cc:323] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:26.923539 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet e3de96a6f20142c0a00ff9ee12bcc86c from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:26.924109 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet e3de96a6f20142c0a00ff9ee12bcc86c from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-e3de96a6f20142c0a00ff9ee12bcc86c
I20260709 12:23:26.928555 31773 tablet_copy_client.cc:806] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:26.929343 31773 tablet_copy_client.cc:670] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:26.934612 31773 tablet_copy_client.cc:538] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:26.943675 31002 tablet_copy_source_session.cc:215] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:26.943694 31773 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:26.947016 31776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3de96a6f20142c0a00ff9ee12bcc86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:26.968402 31776 tablet_copy_client.cc:806] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:26.969007 31776 tablet_copy_client.cc:670] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:27.002451 31773 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:27.003400 31773 tablet_bootstrap.cc:492] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:27.004091 31773 ts_tablet_manager.cc:1403] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.061s	user 0.028s	sys 0.008s
I20260709 12:23:27.007202 31773 raft_consensus.cc:359] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:27.008046 31773 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:27.008736 31773 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:27.011176 31773 ts_tablet_manager.cc:1434] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:27.014541 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-563f5eec53c14273959c9883d2423cc9 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:27.014961 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-563f5eec53c14273959c9883d2423cc9 on tablet 563f5eec53c14273959c9883d2423cc9 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:27.037768 31776 tablet_copy_client.cc:538] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:27.045295 31776 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:27.072090 31776 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:27.072992 31776 tablet_bootstrap.cc:492] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:27.073711 31776 ts_tablet_manager.cc:1403] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.029s	user 0.020s	sys 0.007s
I20260709 12:23:27.077404 31776 raft_consensus.cc:359] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:27.078383 31776 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:27.079138 31776 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:27.082394 31776 ts_tablet_manager.cc:1434] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.001s
I20260709 12:23:27.084296 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-e3de96a6f20142c0a00ff9ee12bcc86c received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:27.084744 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-e3de96a6f20142c0a00ff9ee12bcc86c on tablet e3de96a6f20142c0a00ff9ee12bcc86c with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:27.114576 31776 ts_tablet_manager.cc:933] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:27.120465 31776 tablet_copy_client.cc:323] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:27.122067 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet b8ba55d3e2574f7a890f7a956ba4e74f from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:27.122677 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet b8ba55d3e2574f7a890f7a956ba4e74f from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-b8ba55d3e2574f7a890f7a956ba4e74f
I20260709 12:23:27.135581 31002 tablet_copy_source_session.cc:215] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:27.139828 31776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ba55d3e2574f7a890f7a956ba4e74f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:27.162132 31776 tablet_copy_client.cc:806] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:27.162744 31776 tablet_copy_client.cc:670] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:27.168576 31776 tablet_copy_client.cc:538] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:27.177537 31776 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:27.217669 31698 raft_consensus.cc:1217] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:27.224541 31776 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:27.225456 31776 tablet_bootstrap.cc:492] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:27.226164 31776 ts_tablet_manager.cc:1403] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.021s
I20260709 12:23:27.229441 31776 raft_consensus.cc:359] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:27.230383 31776 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:27.231040 31776 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:27.233703 31776 ts_tablet_manager.cc:1434] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:27.240978 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-b8ba55d3e2574f7a890f7a956ba4e74f received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:27.241375 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-b8ba55d3e2574f7a890f7a956ba4e74f on tablet b8ba55d3e2574f7a890f7a956ba4e74f with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:27.352196 31698 raft_consensus.cc:1217] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:27.439229 31698 raft_consensus.cc:1217] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:23:27.622834 31698 raft_consensus.cc:1217] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:27.632539 31752 raft_consensus.cc:1064] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:27.637341 31752 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:27.644734 31064 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.647063 31698 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.649009 31536 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.650750 31536 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.647022 30987 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.657330 31536 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.715058 31697 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:27.712888 31536 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:27.731918 30987 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:27.741791 31064 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:27.761142 30887 catalog_manager.cc:5697] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:27.807044 31135 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:27.813570 31064 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:27.815032 31697 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:27.814906 31592 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:27.817087 31786 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:27.827709 31592 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:27.834419 31697 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:27.834841 31064 raft_consensus.cc:2955] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:27.838927 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9bb92178abcc4ef385f771bd04448a5b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:27.850085 30885 catalog_manager.cc:5697] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.889783 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 9bb92178abcc4ef385f771bd04448a5b with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:27.904470 31791 tablet_replica.cc:333] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:27.905972 31791 raft_consensus.cc:2243] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:27.906715 31791 raft_consensus.cc:2272] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:27.911481 31791 ts_tablet_manager.cc:1916] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:27.911954 31592 raft_consensus.cc:1064] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:27.915132 31592 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:27.930135 31064 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.930534 31697 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.931483 31752 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.932331 31786 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.933770 31791 ts_tablet_manager.cc:1929] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:27.934280 31791 log.cc:1199] T 9bb92178abcc4ef385f771bd04448a5b P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/9bb92178abcc4ef385f771bd04448a5b
I20260709 12:23:27.935930 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 9bb92178abcc4ef385f771bd04448a5b (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:27.936334 30987 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.938596 31592 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.970901 31735 raft_consensus.cc:1064] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:27.973421 31735 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:28.013765 31613 raft_consensus.cc:1064] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:28.016812 31613 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:28.023692 31698 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:28.024268 31140 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:28.026803 31734 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:28.029330 31734 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:23:28.059638 31067 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:28.060993 31797 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:23:28.068975 31795 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:23:28.081529 31696 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:28.083184 31064 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:28.084633 31797 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:28.085737 31592 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.087380 31140 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:28.090838 31797 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:28.092604 30989 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.096084 31798 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.097271 31062 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.099675 31697 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.124717 31797 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:28.127642 30885 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.130348 30887 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:23:28.140812 31800 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:23:28.145251 31136 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.147202 31067 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.156047 31698 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.165020 31734 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.171991 31696 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.181087 31067 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.182067 31140 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.186672 31135 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:28.195350 30888 catalog_manager.cc:5697] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.199612 30987 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:28.201990 31752 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:28.208328 30989 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:28.211299 31536 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.213615 30985 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.221386 31698 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:28.222692 31064 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:28.225536 31735 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:28.200506 31696 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:28.229343 31536 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:28.229888 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 563f5eec53c14273959c9883d2423cc9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:28.236027 31698 raft_consensus.cc:2955] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.240299 31613 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:28.255755 31735 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.257072 31697 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.247596 30985 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:28.264052 31140 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:28.265268 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e3de96a6f20142c0a00ff9ee12bcc86c with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:28.266749 31064 raft_consensus.cc:2955] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.275857 31734 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:28.284333 31696 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:28.284938 30888 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.290650 31613 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:28.293076 31797 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.297911 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet 563f5eec53c14273959c9883d2423cc9 with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:28.306202 31140 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.307511 30887 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 4 to 5, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.324013 31696 raft_consensus.cc:2955] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:28.328152 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8ba55d3e2574f7a890f7a956ba4e74f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:28.339236 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet e3de96a6f20142c0a00ff9ee12bcc86c with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:28.365645 31806 tablet_replica.cc:333] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:28.392891 30887 catalog_manager.cc:5697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:28.401830 31806 raft_consensus.cc:2243] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:28.402634 31806 raft_consensus.cc:2272] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:28.406134 31806 ts_tablet_manager.cc:1916] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:28.412340 31808 tablet_replica.cc:333] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:28.416394 31808 raft_consensus.cc:2243] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:28.417178 31808 raft_consensus.cc:2272] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:28.420812 31808 ts_tablet_manager.cc:1916] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:28.437784 31808 ts_tablet_manager.cc:1929] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:23:28.438319 31808 log.cc:1199] T e3de96a6f20142c0a00ff9ee12bcc86c P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/e3de96a6f20142c0a00ff9ee12bcc86c
I20260709 12:23:28.442236 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet e3de96a6f20142c0a00ff9ee12bcc86c (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:28.452296 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet b8ba55d3e2574f7a890f7a956ba4e74f with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:28.454983 31806 ts_tablet_manager.cc:1929] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:28.455518 31806 log.cc:1199] T 563f5eec53c14273959c9883d2423cc9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/563f5eec53c14273959c9883d2423cc9
I20260709 12:23:28.457237 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet 563f5eec53c14273959c9883d2423cc9 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:28.457528 31806 tablet_replica.cc:333] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:28.458412 31806 raft_consensus.cc:2243] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:28.459049 31806 raft_consensus.cc:2272] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:28.462491 31806 ts_tablet_manager.cc:1916] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:28.479822 31806 ts_tablet_manager.cc:1929] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:28.480218 31806 log.cc:1199] T b8ba55d3e2574f7a890f7a956ba4e74f P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/b8ba55d3e2574f7a890f7a956ba4e74f
I20260709 12:23:28.481707 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet b8ba55d3e2574f7a890f7a956ba4e74f (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:29.528627 30985 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:29.535343 31140 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:29.535627 30958 consensus_peers.cc:597] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0afdb545bb8f485993259007a00c1841. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:29.536092 31064 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:29.537352 31734 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.538444 31797 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.550299 31613 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.552000 31140 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.551905 31064 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.571054 31139 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:29.574731 30887 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.593624 31064 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:29.595947 31807 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.602326 30985 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:29.602911 31109 consensus_peers.cc:597] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff2c202e27d24d1baad8ddf5d9db4391. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:29.604061 31536 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.617036 31536 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.620509 30987 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:23:29.665285 31815 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 12:23:29.676287 31816 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.056s	user 0.012s	sys 0.017s
I20260709 12:23:29.701856 31139 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:29.709404 30885 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.726837 31064 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.736529 31062 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:29.740607 31109 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 76fe9dfd8b4c4785a354982ad7e1faa4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:29.747068 31536 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.747092 30987 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:29.752789 31807 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.764556 31827 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.776640 30989 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.785625 31064 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.794698 30885 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.841166 31062 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:29.849822 31139 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:29.851805 31819 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.859087 30989 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:29.861100 31819 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:29.867774 31646 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.877540 31140 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:29.889521 30886 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.937115 30987 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:23:29.938486 31033 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82399103216c4d5e8d2acdbbf3cee84d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:30.076179 31837 ts_tablet_manager.cc:933] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:30.078187 31837 tablet_copy_client.cc:323] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:30.079687 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet 0afdb545bb8f485993259007a00c1841 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:30.080178 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet 0afdb545bb8f485993259007a00c1841 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-0afdb545bb8f485993259007a00c1841
I20260709 12:23:30.085445 31002 tablet_copy_source_session.cc:215] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:30.098201 31837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0afdb545bb8f485993259007a00c1841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:30.122669 31837 tablet_copy_client.cc:806] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:30.123287 31837 tablet_copy_client.cc:670] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:30.129730 31837 tablet_copy_client.cc:538] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:30.138598 31837 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:30.163795 31837 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:30.164510 31837 tablet_bootstrap.cc:492] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:30.165015 31837 ts_tablet_manager.cc:1403] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.009s
I20260709 12:23:30.167483 31837 raft_consensus.cc:359] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:30.168284 31837 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:30.168977 31837 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:30.291574 31845 ts_tablet_manager.cc:933] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:30.295958 31845 tablet_copy_client.cc:323] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:30.302011 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet ff2c202e27d24d1baad8ddf5d9db4391 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:30.302683 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet ff2c202e27d24d1baad8ddf5d9db4391 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-ff2c202e27d24d1baad8ddf5d9db4391
I20260709 12:23:30.310813 31154 tablet_copy_source_session.cc:215] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:30.302618 31846 ts_tablet_manager.cc:933] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:30.319190 31845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2c202e27d24d1baad8ddf5d9db4391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:30.325413 31846 tablet_copy_client.cc:323] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:30.327199 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 76fe9dfd8b4c4785a354982ad7e1faa4 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:30.327746 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 76fe9dfd8b4c4785a354982ad7e1faa4 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-76fe9dfd8b4c4785a354982ad7e1faa4
W20260709 12:23:30.330837 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90043us
I20260709 12:23:30.335888 31154 tablet_copy_source_session.cc:215] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:23:30.337033 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 67907.8us
I20260709 12:23:30.338595 31841 ts_tablet_manager.cc:933] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:30.352046 31846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76fe9dfd8b4c4785a354982ad7e1faa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:30.356719 31837 ts_tablet_manager.cc:1434] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.191s	user 0.035s	sys 0.061s
I20260709 12:23:30.368434 31845 tablet_copy_client.cc:806] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:30.369004 31845 tablet_copy_client.cc:670] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:30.373260 31845 tablet_copy_client.cc:538] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:30.380107 31841 tablet_copy_client.cc:323] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:30.381332 31845 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:30.382933 31846 tablet_copy_client.cc:806] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:30.392772 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-0afdb545bb8f485993259007a00c1841 received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:30.392050 31846 tablet_copy_client.cc:670] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:30.393193 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-0afdb545bb8f485993259007a00c1841 on tablet 0afdb545bb8f485993259007a00c1841 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:30.399017 31846 tablet_copy_client.cc:538] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:30.432627 31846 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:30.443231 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 82399103216c4d5e8d2acdbbf3cee84d from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:30.443813 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 82399103216c4d5e8d2acdbbf3cee84d from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-82399103216c4d5e8d2acdbbf3cee84d
I20260709 12:23:30.457284 31078 tablet_copy_source_session.cc:215] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:30.462922 31841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82399103216c4d5e8d2acdbbf3cee84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:30.465260 31846 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:30.466361 31846 tablet_bootstrap.cc:492] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:30.467232 31846 ts_tablet_manager.cc:1403] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.017s
I20260709 12:23:30.470697 31846 raft_consensus.cc:359] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:30.466391 31845 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:30.471592 31846 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:30.472566 31845 tablet_bootstrap.cc:492] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:30.472400 31846 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:30.473383 31845 ts_tablet_manager.cc:1403] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.092s	user 0.023s	sys 0.000s
I20260709 12:23:30.478619 31846 ts_tablet_manager.cc:1434] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 12:23:30.480226 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-76fe9dfd8b4c4785a354982ad7e1faa4 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:30.480654 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-76fe9dfd8b4c4785a354982ad7e1faa4 on tablet 76fe9dfd8b4c4785a354982ad7e1faa4 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:30.481379 31845 raft_consensus.cc:359] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:30.482375 31845 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:30.483134 31845 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:30.499325 31845 ts_tablet_manager.cc:1434] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.004s
I20260709 12:23:30.499989 31841 tablet_copy_client.cc:806] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:30.500931 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-ff2c202e27d24d1baad8ddf5d9db4391 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:30.501303 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-ff2c202e27d24d1baad8ddf5d9db4391 on tablet ff2c202e27d24d1baad8ddf5d9db4391 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:30.500581 31841 tablet_copy_client.cc:670] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:30.506532 31841 tablet_copy_client.cc:538] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:30.515949 31841 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:30.562129 31841 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:30.563256 31841 tablet_bootstrap.cc:492] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:30.563939 31841 ts_tablet_manager.cc:1403] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.048s	user 0.027s	sys 0.021s
I20260709 12:23:30.563841 31696 raft_consensus.cc:1217] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:30.568253 31841 raft_consensus.cc:359] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:30.569085 31841 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:30.569890 31841 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:30.573400 31841 ts_tablet_manager.cc:1434] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:30.575163 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-82399103216c4d5e8d2acdbbf3cee84d received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:30.575587 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-82399103216c4d5e8d2acdbbf3cee84d on tablet 82399103216c4d5e8d2acdbbf3cee84d with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:30.699329 31696 raft_consensus.cc:1217] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:30.728288 31696 raft_consensus.cc:1217] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:30.893106 31696 raft_consensus.cc:1217] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:31.170789 31827 raft_consensus.cc:1064] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:31.175899 31827 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:31.195648 31696 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.194203 31064 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.198400 31536 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.204521 31823 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.212299 30989 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.219609 31823 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.253670 31823 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.263329 30989 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.270816 31062 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.269681 31696 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.290093 30886 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:31.304514 31797 raft_consensus.cc:1064] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:31.306589 31797 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:31.310410 31821 raft_consensus.cc:1064] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:31.315286 31140 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:31.313040 31821 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:31.327206 31062 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:31.335528 31140 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.339267 31062 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.339542 31860 raft_consensus.cc:1064] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:31.340816 31843 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.343992 31696 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:31.347432 31536 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:31.353463 31843 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.359419 31861 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:31.368884 31698 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.370604 31870 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.376843 31827 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.380357 31064 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.412938 30887 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.419867 31860 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:31.443656 31696 raft_consensus.cc:2955] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.460956 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet ff2c202e27d24d1baad8ddf5d9db4391 with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:31.484504 31797 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.497532 31698 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.528499 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ff2c202e27d24d1baad8ddf5d9db4391 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:31.572110 31873 tablet_replica.cc:333] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:31.573244 31873 raft_consensus.cc:2243] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:31.573987 31873 raft_consensus.cc:2272] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:31.583491 31873 ts_tablet_manager.cc:1916] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:31.603991 31696 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.605679 31139 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.605839 31696 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.605971 31064 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.606439 31819 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.606881 30887 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:31.607582 31860 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:31.609779 31536 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.610704 30987 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:31.615343 31536 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.617870 31877 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:31.618280 31139 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:23:31.668300 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52026.2us
I20260709 12:23:31.710067 30986 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:31.726300 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45340.1us
I20260709 12:23:31.743872 31819 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:31.776142 31064 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.795354 31878 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.801484 30987 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.837921 30986 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:31.845194 31062 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:31.846894 31888 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:31.848624 31873 ts_tablet_manager.cc:1929] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:31.849159 31873 log.cc:1199] T ff2c202e27d24d1baad8ddf5d9db4391 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/ff2c202e27d24d1baad8ddf5d9db4391
W20260709 12:23:31.854992 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 80192.7us
I20260709 12:23:31.860874 31875 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.861976 31062 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.862982 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet ff2c202e27d24d1baad8ddf5d9db4391 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:31.869292 31696 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:31.870756 31797 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:31.873554 31698 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.876039 31696 raft_consensus.cc:2955] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.886576 31818 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.890879 30887 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.892645 30987 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.896509 31064 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.898984 31698 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.910899 31139 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.921015 30886 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:31.923781 30887 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.933689 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0afdb545bb8f485993259007a00c1841 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:31.936183 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 0afdb545bb8f485993259007a00c1841 with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:31.940491 31062 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:31.946715 31698 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:31.947880 30987 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:31.949246 31818 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:31.949749 31139 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:31.952308 31818 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:31.961556 31872 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.971371 31898 tablet_replica.cc:333] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:31.972764 31898 raft_consensus.cc:2243] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:31.973462 31898 raft_consensus.cc:2272] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:31.974014 30987 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.977620 31898 ts_tablet_manager.cc:1916] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:31.980087 31698 raft_consensus.cc:2955] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:31.982963 31696 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:31.983987 30986 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:31.985224 31860 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:31.986753 31881 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:32.028185 31898 ts_tablet_manager.cc:1929] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:32.028708 30888 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:32.031111 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 82399103216c4d5e8d2acdbbf3cee84d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:32.036793 31878 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:32.028752 31898 log.cc:1199] T 0afdb545bb8f485993259007a00c1841 P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/0afdb545bb8f485993259007a00c1841
I20260709 12:23:32.041637 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 0afdb545bb8f485993259007a00c1841 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:32.047256 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 76fe9dfd8b4c4785a354982ad7e1faa4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:32.047816 30986 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:32.054956 31698 raft_consensus.cc:2955] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:32.057116 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 82399103216c4d5e8d2acdbbf3cee84d with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:32.066743 31898 tablet_replica.cc:333] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:32.067831 31898 raft_consensus.cc:2243] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:32.068625 31898 raft_consensus.cc:2272] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:32.072368 31898 ts_tablet_manager.cc:1916] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:32.087213 30888 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:32.112416 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet 76fe9dfd8b4c4785a354982ad7e1faa4 with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:32.115302 31898 ts_tablet_manager.cc:1929] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:32.115876 31898 log.cc:1199] T 82399103216c4d5e8d2acdbbf3cee84d P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/82399103216c4d5e8d2acdbbf3cee84d
I20260709 12:23:32.117755 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 82399103216c4d5e8d2acdbbf3cee84d (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:32.119788 31899 tablet_replica.cc:333] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:32.120560 31899 raft_consensus.cc:2243] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:32.121258 31899 raft_consensus.cc:2272] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:32.124250 31899 ts_tablet_manager.cc:1916] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:32.135964 31899 ts_tablet_manager.cc:1929] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:32.136345 31899 log.cc:1199] T 76fe9dfd8b4c4785a354982ad7e1faa4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/76fe9dfd8b4c4785a354982ad7e1faa4
I20260709 12:23:32.137605 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet 76fe9dfd8b4c4785a354982ad7e1faa4 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:33.180058 31142 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.188350 31068 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:33.189675 31878 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:33.190534 31109 consensus_peers.cc:597] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ff17d7bb34443adb04e437ed0079e51. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:33.190773 30986 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:33.192194 31861 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.199132 31855 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.200048 31064 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.211894 30989 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.221798 30887 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.227540 30986 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.255071 31142 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:33.255927 31064 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:33.256261 31797 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:33.259419 30958 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e81e4942d094afda44355ba1d9c7735. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:33.264441 31875 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.279186 31910 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.291409 31139 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.293815 31064 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.309434 31064 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.309547 30885 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:23:33.316202 31033 consensus_peers.cc:597] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 330f8038ee4e498ebefd9d773f4e7199. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:33.315490 31139 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:33.316845 30986 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:33.317831 31907 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.318770 31818 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.331482 31821 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.334898 31142 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.342991 30986 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.351609 31135 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.350750 30888 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.366736 30986 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:33.367020 31109 consensus_peers.cc:597] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4950edf9dfb9412981a0b325bf866deb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:33.367993 31878 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.370237 31064 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:33.373135 31878 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.378821 31878 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.380419 30986 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.399991 30885 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.418944 31064 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:33.698051 31920 ts_tablet_manager.cc:933] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:33.703833 31920 tablet_copy_client.cc:323] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:33.705713 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet 8e81e4942d094afda44355ba1d9c7735 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:33.706235 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet 8e81e4942d094afda44355ba1d9c7735 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-8e81e4942d094afda44355ba1d9c7735
I20260709 12:23:33.713421 31002 tablet_copy_source_session.cc:215] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:33.716544 31920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e81e4942d094afda44355ba1d9c7735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:33.730423 31920 tablet_copy_client.cc:806] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:33.730975 31920 tablet_copy_client.cc:670] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:33.735934 31920 tablet_copy_client.cc:538] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:33.742684 31920 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:33.758526 31920 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:33.759197 31920 tablet_bootstrap.cc:492] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:33.759765 31920 ts_tablet_manager.cc:1403] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 12:23:33.762369 31920 raft_consensus.cc:359] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.763207 31920 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:33.764343 31920 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.767588 31920 ts_tablet_manager.cc:1434] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:23:33.784922 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-8e81e4942d094afda44355ba1d9c7735 received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:33.785368 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-8e81e4942d094afda44355ba1d9c7735 on tablet 8e81e4942d094afda44355ba1d9c7735 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:33.799968 31920 ts_tablet_manager.cc:933] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:33.801581 31920 tablet_copy_client.cc:323] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:33.803000 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 4ff17d7bb34443adb04e437ed0079e51 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:33.803498 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 4ff17d7bb34443adb04e437ed0079e51 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-4ff17d7bb34443adb04e437ed0079e51
I20260709 12:23:33.810806 31154 tablet_copy_source_session.cc:215] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:33.814509 31920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff17d7bb34443adb04e437ed0079e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:33.828192 31920 tablet_copy_client.cc:806] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:33.828819 31920 tablet_copy_client.cc:670] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:33.832216 31920 tablet_copy_client.cc:538] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:33.834496 31925 ts_tablet_manager.cc:933] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:33.841051 31925 tablet_copy_client.cc:323] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:33.841388 31920 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:33.843055 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 4950edf9dfb9412981a0b325bf866deb from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:33.843570 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 4950edf9dfb9412981a0b325bf866deb from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-4950edf9dfb9412981a0b325bf866deb
I20260709 12:23:33.850445 31154 tablet_copy_source_session.cc:215] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:33.853288 31925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4950edf9dfb9412981a0b325bf866deb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:33.868940 31920 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:33.869865 31920 tablet_bootstrap.cc:492] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:33.870551 31920 ts_tablet_manager.cc:1403] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.008s
I20260709 12:23:33.873481 31920 raft_consensus.cc:359] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.874326 31920 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:33.875298 31925 tablet_copy_client.cc:806] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:33.874996 31920 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.875869 31925 tablet_copy_client.cc:670] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:33.877436 31920 ts_tablet_manager.cc:1434] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:33.879786 31925 tablet_copy_client.cc:538] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:33.882253 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-4ff17d7bb34443adb04e437ed0079e51 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:33.882681 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-4ff17d7bb34443adb04e437ed0079e51 on tablet 4ff17d7bb34443adb04e437ed0079e51 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:33.889901 31925 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:33.901616 31928 ts_tablet_manager.cc:933] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:33.910576 31928 tablet_copy_client.cc:323] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:33.912849 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 330f8038ee4e498ebefd9d773f4e7199 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:33.913369 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 330f8038ee4e498ebefd9d773f4e7199 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-330f8038ee4e498ebefd9d773f4e7199
I20260709 12:23:33.921629 31078 tablet_copy_source_session.cc:215] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:33.923099 31925 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:33.926708 31928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 330f8038ee4e498ebefd9d773f4e7199. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:33.934571 31925 tablet_bootstrap.cc:492] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:33.935372 31925 ts_tablet_manager.cc:1403] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.017s
I20260709 12:23:33.939502 31925 raft_consensus.cc:359] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.940347 31925 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:33.941286 31925 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.949080 31928 tablet_copy_client.cc:806] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:33.949749 31928 tablet_copy_client.cc:670] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:33.950492 31925 ts_tablet_manager.cc:1434] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.006s
I20260709 12:23:33.952132 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-4950edf9dfb9412981a0b325bf866deb received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:33.952541 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-4950edf9dfb9412981a0b325bf866deb on tablet 4950edf9dfb9412981a0b325bf866deb with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:33.953548 31928 tablet_copy_client.cc:538] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:33.962069 31928 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:33.981518 31928 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:33.982391 31928 tablet_bootstrap.cc:492] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:33.982983 31928 ts_tablet_manager.cc:1403] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.021s	user 0.023s	sys 0.000s
I20260709 12:23:33.985791 31928 raft_consensus.cc:359] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.986641 31928 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:33.987336 31928 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:33.989682 31928 ts_tablet_manager.cc:1434] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:33.991503 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-330f8038ee4e498ebefd9d773f4e7199 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:33.991917 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-330f8038ee4e498ebefd9d773f4e7199 on tablet 330f8038ee4e498ebefd9d773f4e7199 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:34.076026 31698 raft_consensus.cc:1217] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:34.216591 31698 raft_consensus.cc:1217] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:34.322180 31698 raft_consensus.cc:1217] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:34.503108 31698 raft_consensus.cc:1217] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:34.642009 31797 raft_consensus.cc:1064] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:34.674767 31797 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:34.734799 31698 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.735810 31142 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.738139 31934 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:34.740473 31934 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:34.748919 31062 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.750624 31912 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:34.831959 31938 raft_consensus.cc:1064] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
W20260709 12:23:34.834257 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 64226.6us
I20260709 12:23:34.840436 31797 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:34.842056 31938 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:34.865115 31936 raft_consensus.cc:1064] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:34.869488 31936 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:34.877774 31696 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:34.880070 31142 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:23:34.906116 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 77263us
I20260709 12:23:34.916100 30888 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:34.921343 31062 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:34.929953 31064 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.930119 30986 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.931715 31940 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:34.932724 31855 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:34.942654 31696 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.951121 31068 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.952129 31698 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.952687 31938 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:34.953356 31940 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:34.955862 30987 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:34.955742 31939 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:34.959522 31855 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:34.976238 31945 raft_consensus.cc:1064] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:34.979825 31945 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:35.019670 31939 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.042958 31698 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:23:35.161548 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75338us
I20260709 12:23:35.210942 31698 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:35.211042 31135 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:35.213999 31061 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.221146 30987 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:35.227936 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 77949.7us
I20260709 12:23:35.235261 30986 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.233461 31855 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.248764 30989 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:35.251266 31698 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.255384 30990 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.242295 31957 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:35.262413 31932 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:35.264577 31945 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:35.239817 30888 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.266767 31067 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:35.268136 31912 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:35.275964 31797 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.285772 31698 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:35.287370 31062 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.290205 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8e81e4942d094afda44355ba1d9c7735 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:35.295660 30886 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.308286 31946 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:35.313915 31696 raft_consensus.cc:2955] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.343538 31142 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:35.348393 30885 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.356891 31067 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:35.360285 31861 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:35.363250 31060 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.370132 31061 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.370901 31903 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.382176 31696 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:35.389958 31945 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.391275 31142 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.392701 31940 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:35.397090 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e81e4942d094afda44355ba1d9c7735 with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:35.401702 31696 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.403240 31697 raft_consensus.cc:2955] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.416285 31144 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:35.430051 30990 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:35.430452 31697 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:35.432505 30886 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.433363 31940 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:35.434428 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4ff17d7bb34443adb04e437ed0079e51 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:35.438529 31855 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:35.447785 31964 tablet_replica.cc:333] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:35.448925 31964 raft_consensus.cc:2243] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:35.449704 31964 raft_consensus.cc:2272] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:35.451809 30885 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.452968 31861 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.454841 31964 ts_tablet_manager.cc:1916] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:35.456377 30990 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.465312 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ff17d7bb34443adb04e437ed0079e51 with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:35.480088 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4950edf9dfb9412981a0b325bf866deb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:35.484963 31965 tablet_replica.cc:333] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:35.485946 30990 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.509742 31697 raft_consensus.cc:2955] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.547978 31965 raft_consensus.cc:2243] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:35.548722 31965 raft_consensus.cc:2272] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:35.550714 31964 ts_tablet_manager.cc:1929] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:35.551358 31964 log.cc:1199] T 8e81e4942d094afda44355ba1d9c7735 P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/8e81e4942d094afda44355ba1d9c7735
I20260709 12:23:35.552260 31965 ts_tablet_manager.cc:1916] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:35.552237 30886 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.562856 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet 4950edf9dfb9412981a0b325bf866deb with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:35.565371 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 8e81e4942d094afda44355ba1d9c7735 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:35.568054 31062 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:35.573267 31697 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:35.573433 31966 tablet_replica.cc:333] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:35.574330 31135 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:35.574476 31966 raft_consensus.cc:2243] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:35.575196 31966 raft_consensus.cc:2272] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:35.576057 31957 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:35.577816 31957 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:35.583843 31966 ts_tablet_manager.cc:1916] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:35.592558 31945 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.595095 31135 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.597218 31697 raft_consensus.cc:2955] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.599982 31966 ts_tablet_manager.cc:1929] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:35.600440 31966 log.cc:1199] T 4950edf9dfb9412981a0b325bf866deb P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/4950edf9dfb9412981a0b325bf866deb
I20260709 12:23:35.602488 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet 4950edf9dfb9412981a0b325bf866deb (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:35.614733 31965 ts_tablet_manager.cc:1929] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:35.615191 31965 log.cc:1199] T 4ff17d7bb34443adb04e437ed0079e51 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/4ff17d7bb34443adb04e437ed0079e51
I20260709 12:23:35.616933 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 4ff17d7bb34443adb04e437ed0079e51 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:35.621232 30886 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:35.623157 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 330f8038ee4e498ebefd9d773f4e7199 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:35.637167 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 330f8038ee4e498ebefd9d773f4e7199 with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:35.638238 31965 tablet_replica.cc:333] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:35.639400 31965 raft_consensus.cc:2243] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:35.640296 31965 raft_consensus.cc:2272] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:35.643951 31965 ts_tablet_manager.cc:1916] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:35.657271 31965 ts_tablet_manager.cc:1929] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:35.657675 31965 log.cc:1199] T 330f8038ee4e498ebefd9d773f4e7199 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/330f8038ee4e498ebefd9d773f4e7199
I20260709 12:23:35.659199 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 330f8038ee4e498ebefd9d773f4e7199 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:36.708559 31135 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:36.714308 31062 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:36.714708 30990 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:36.715101 31109 consensus_peers.cc:597] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c6a2f53447447459d88ab978179c65c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:36.715652 31903 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:36.716499 31968 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:36.724839 31903 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.726509 31062 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.736740 30989 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.745440 30886 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.747391 30990 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:36.766585 31135 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:36.768314 30958 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72d9d9cc8be6411883e9f3f17ef865eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:36.769325 31912 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:36.776796 31062 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:36.778514 31797 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:36.778155 31912 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.782436 31135 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.784202 31060 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.798126 30990 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:36.808386 30887 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.813602 31060 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:36.815135 30958 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60b6187f32774daba89763099dabeb2a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:36.816943 31912 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:36.823315 31135 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:36.825786 31797 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:36.833873 31797 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.835626 31061 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.857725 31144 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.858155 31062 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:36.877730 30885 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.905653 31144 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:36.907110 31937 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:36.912330 30987 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:36.915560 31033 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 544dffe603ca4dce9d6697ea6b15c925. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:36.916386 31957 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:36.918205 31957 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.938726 31135 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.940845 30987 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:36.946468 30886 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:37.148567 31993 ts_tablet_manager.cc:933] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:37.150974 31993 tablet_copy_client.cc:323] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:37.152671 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 0c6a2f53447447459d88ab978179c65c from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:37.153208 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 0c6a2f53447447459d88ab978179c65c from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-0c6a2f53447447459d88ab978179c65c
I20260709 12:23:37.165410 31154 tablet_copy_source_session.cc:215] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:37.175132 31993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c6a2f53447447459d88ab978179c65c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:37.191728 31993 tablet_copy_client.cc:806] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:37.192271 31993 tablet_copy_client.cc:670] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:37.196077 31993 tablet_copy_client.cc:538] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:37.203236 31993 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:37.221207 31993 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:37.222105 31993 tablet_bootstrap.cc:492] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:37.222822 31993 ts_tablet_manager.cc:1403] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 12:23:37.225291 31993 raft_consensus.cc:359] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:37.225958 31993 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:37.227082 31993 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:37.230402 31993 ts_tablet_manager.cc:1434] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:23:37.232061 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-0c6a2f53447447459d88ab978179c65c received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:37.232535 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-0c6a2f53447447459d88ab978179c65c on tablet 0c6a2f53447447459d88ab978179c65c with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:37.234264 31997 ts_tablet_manager.cc:933] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:37.237210 31993 ts_tablet_manager.cc:933] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:37.243484 31997 tablet_copy_client.cc:323] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:37.245378 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet 60b6187f32774daba89763099dabeb2a from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:37.245877 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet 60b6187f32774daba89763099dabeb2a from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-60b6187f32774daba89763099dabeb2a
I20260709 12:23:37.253377 31002 tablet_copy_source_session.cc:215] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:37.254489 31993 tablet_copy_client.cc:323] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:37.256395 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet 72d9d9cc8be6411883e9f3f17ef865eb from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:37.256875 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet 72d9d9cc8be6411883e9f3f17ef865eb from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-72d9d9cc8be6411883e9f3f17ef865eb
I20260709 12:23:37.257330 31997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b6187f32774daba89763099dabeb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:37.263922 31002 tablet_copy_source_session.cc:215] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:37.266857 31993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72d9d9cc8be6411883e9f3f17ef865eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:37.272173 31997 tablet_copy_client.cc:806] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:37.272760 31997 tablet_copy_client.cc:670] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:37.278142 31997 tablet_copy_client.cc:538] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:37.282560 31993 tablet_copy_client.cc:806] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:37.283229 31993 tablet_copy_client.cc:670] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:37.287683 31993 tablet_copy_client.cc:538] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:37.288676 31997 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:37.302763 31993 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:37.313395 31997 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:37.314411 31997 tablet_bootstrap.cc:492] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:37.317164 31997 ts_tablet_manager.cc:1403] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.029s	user 0.022s	sys 0.005s
I20260709 12:23:37.321269 31997 raft_consensus.cc:359] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:37.322410 31997 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:37.323076 31997 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:37.325595 31997 ts_tablet_manager.cc:1434] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:23:37.327543 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-60b6187f32774daba89763099dabeb2a received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:37.327961 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-60b6187f32774daba89763099dabeb2a on tablet 60b6187f32774daba89763099dabeb2a with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:37.329074 31993 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:37.333614 31993 tablet_bootstrap.cc:492] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:37.335233 31993 ts_tablet_manager.cc:1403] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.032s	user 0.023s	sys 0.004s
I20260709 12:23:37.340061 31993 raft_consensus.cc:359] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:37.341856 31993 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:37.342429 31993 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:37.344846 31993 ts_tablet_manager.cc:1434] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:23:37.346452 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-72d9d9cc8be6411883e9f3f17ef865eb received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:37.346853 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-72d9d9cc8be6411883e9f3f17ef865eb on tablet 72d9d9cc8be6411883e9f3f17ef865eb with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:37.437734 31993 ts_tablet_manager.cc:933] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:37.445649 31993 tablet_copy_client.cc:323] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:37.447647 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 544dffe603ca4dce9d6697ea6b15c925 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:37.448191 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 544dffe603ca4dce9d6697ea6b15c925 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-544dffe603ca4dce9d6697ea6b15c925
I20260709 12:23:37.454923 31078 tablet_copy_source_session.cc:215] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:37.457741 31993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544dffe603ca4dce9d6697ea6b15c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:37.473619 31993 tablet_copy_client.cc:806] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:37.474117 31993 tablet_copy_client.cc:670] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:37.477470 31993 tablet_copy_client.cc:538] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:37.485638 31993 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:37.499729 31993 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:37.500362 31993 tablet_bootstrap.cc:492] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:37.500818 31993 ts_tablet_manager.cc:1403] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:23:37.502738 31993 raft_consensus.cc:359] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:37.503456 31993 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:37.504019 31993 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:37.506743 31993 ts_tablet_manager.cc:1434] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:23:37.508191 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-544dffe603ca4dce9d6697ea6b15c925 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:37.508663 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-544dffe603ca4dce9d6697ea6b15c925 on tablet 544dffe603ca4dce9d6697ea6b15c925 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:37.683636 31697 raft_consensus.cc:1217] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:37.717118 31697 raft_consensus.cc:1217] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:37.817220 31697 raft_consensus.cc:1217] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:37.824714 31697 raft_consensus.cc:1217] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:38.196610 31990 raft_consensus.cc:1064] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:38.199162 31990 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:38.206146 31135 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.207731 31696 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.208240 30987 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.209565 31990 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.210314 31977 raft_consensus.cc:1064] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:38.212787 31977 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:38.213521 31957 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.218091 32006 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:38.232681 32004 raft_consensus.cc:1064] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:38.236286 31697 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.237214 32004 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:38.237844 31976 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.259778 31062 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.262669 31912 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.269039 31698 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.271631 31061 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.273590 31135 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.275343 31957 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.277406 31940 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.279703 31940 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.281325 31696 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.302332 31135 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.318778 31977 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.322096 30887 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:38.333562 30990 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:23:38.350627 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 77607.3us
I20260709 12:23:38.358970 30985 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.360355 31972 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.363428 31977 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.367291 31064 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.385335 32015 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.395921 31061 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.419080 31697 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.433569 30987 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.453682 31697 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.464112 31976 raft_consensus.cc:1064] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:38.484316 31142 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.493403 31976 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:38.522475 31061 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:38.530819 30990 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:38.532193 31957 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:38.552035 31061 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.553542 31912 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.553925 31696 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.555677 32006 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.561266 31135 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.541100 30885 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:38.565469 30885 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.568490 31912 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.569585 31697 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:38.570917 30990 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.574623 31912 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.583065 32025 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:38.586846 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 544dffe603ca4dce9d6697ea6b15c925 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:38.591889 31696 raft_consensus.cc:2955] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.615703 31144 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:38.622349 31696 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:38.624347 32014 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:38.729784 31972 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.744132 31696 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.791764 31061 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:38.814312 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0c6a2f53447447459d88ab978179c65c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:38.850718 32004 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:38.856936 31061 raft_consensus.cc:2955] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.892498 31142 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.757711 31977 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.944705 31696 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.948694 31061 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:38.983675 30990 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:38.983397 30885 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.023809 30887 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.025386 30887 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.026978 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c6a2f53447447459d88ab978179c65c with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:39.039852 31696 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:39.040853 31061 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:39.042289 31976 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.043206 31912 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.050740 32037 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.053589 31061 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.057025 31696 raft_consensus.cc:2955] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.060618 32039 tablet_replica.cc:333] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:39.060736 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 60b6187f32774daba89763099dabeb2a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:39.075065 30989 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:39.070508 32039 raft_consensus.cc:2243] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:39.077332 32039 raft_consensus.cc:2272] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:39.080279 30885 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.085523 31135 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:39.085654 31696 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:39.081097 32039 ts_tablet_manager.cc:1916] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:39.083894 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 544dffe603ca4dce9d6697ea6b15c925 with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:39.090963 32013 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:39.092777 32013 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.100579 31132 tablet_service.cc:1558] Processing DeleteTablet for tablet 60b6187f32774daba89763099dabeb2a with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:39.111277 32040 tablet_replica.cc:333] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:39.111239 32039 ts_tablet_manager.cc:1929] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:39.112416 32039 log.cc:1199] T 0c6a2f53447447459d88ab978179c65c P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/0c6a2f53447447459d88ab978179c65c
I20260709 12:23:39.112540 32040 raft_consensus.cc:2243] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:39.113382 32040 raft_consensus.cc:2272] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:39.114044 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 0c6a2f53447447459d88ab978179c65c (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:39.113265 32034 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.117718 31696 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.118418 31136 raft_consensus.cc:2955] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:39.123572 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 72d9d9cc8be6411883e9f3f17ef865eb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:39.134027 30889 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.140026 32040 ts_tablet_manager.cc:1916] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:39.145097 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet 72d9d9cc8be6411883e9f3f17ef865eb with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:39.146848 32041 tablet_replica.cc:333] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:39.148074 32041 raft_consensus.cc:2243] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:39.148751 32041 raft_consensus.cc:2272] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:39.155347 32041 ts_tablet_manager.cc:1916] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:39.155731 32040 ts_tablet_manager.cc:1929] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:39.158325 32040 log.cc:1199] T 544dffe603ca4dce9d6697ea6b15c925 P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/544dffe603ca4dce9d6697ea6b15c925
I20260709 12:23:39.160351 32040 tablet_replica.cc:333] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:39.160562 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 544dffe603ca4dce9d6697ea6b15c925 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:39.161266 32040 raft_consensus.cc:2243] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:39.162063 32040 raft_consensus.cc:2272] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:39.165578 32040 ts_tablet_manager.cc:1916] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:39.171375 32041 ts_tablet_manager.cc:1929] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:39.171841 32041 log.cc:1199] T 72d9d9cc8be6411883e9f3f17ef865eb P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/72d9d9cc8be6411883e9f3f17ef865eb
I20260709 12:23:39.173527 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet 72d9d9cc8be6411883e9f3f17ef865eb (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:39.177248 32040 ts_tablet_manager.cc:1929] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:39.177551 32040 log.cc:1199] T 60b6187f32774daba89763099dabeb2a P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/60b6187f32774daba89763099dabeb2a
I20260709 12:23:39.178617 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 60b6187f32774daba89763099dabeb2a (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
W20260709 12:23:39.776707 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800039f20 after lost signal to thread 29848
W20260709 12:23:39.777536 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800034600 after lost signal to thread 30924
W20260709 12:23:39.778445 30921 debug-util.cc:398] Leaking SignalData structure 0x7b080031fa60 after lost signal to thread 31016
W20260709 12:23:39.779230 30921 debug-util.cc:398] Leaking SignalData structure 0x7b080026c440 after lost signal to thread 31093
W20260709 12:23:39.779994 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800232420 after lost signal to thread 31168
W20260709 12:23:39.780781 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800286880 after lost signal to thread 31722
I20260709 12:23:40.218182 31061 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:40.224501 31136 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:40.225498 31033 consensus_peers.cc:597] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45809554a739456ea0a596484ede5ac7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:40.226259 30984 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.226605 31989 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.227689 32025 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.235674 32025 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.238879 30984 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.255231 30887 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:40.260022 30989 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:40.244721 31136 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.285440 31135 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.287340 32034 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.287711 31061 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.289249 32034 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:40.297227 30958 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba8a01bf2a514da2805eb75a7c28c412. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:40.318817 32034 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.323563 31061 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.348385 31140 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.380357 31140 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:40.397332 31063 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:40.398797 31109 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 635264f51b2d4410a1da9b76606a6350. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:40.399204 30989 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.400488 32032 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.402274 32032 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.446447 32057 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.447816 30984 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.410142 30889 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.466964 31061 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.503561 30887 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.507004 30984 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
W20260709 12:23:40.517050 30958 consensus_peers.cc:597] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1c1f8d5241540fdb4fb4daa1691260f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:40.518086 31140 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.520566 31061 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.522343 32058 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.525677 32034 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.565583 32058 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.570295 31061 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.575192 31136 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.591969 30887 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:40.787387 32064 ts_tablet_manager.cc:933] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:40.796731 32064 tablet_copy_client.cc:323] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:40.798498 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 45809554a739456ea0a596484ede5ac7 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:40.799033 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 45809554a739456ea0a596484ede5ac7 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-45809554a739456ea0a596484ede5ac7
I20260709 12:23:40.806656 31078 tablet_copy_source_session.cc:215] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:40.810863 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45809554a739456ea0a596484ede5ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:40.829531 32064 tablet_copy_client.cc:806] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:40.830204 32064 tablet_copy_client.cc:670] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:40.834074 32064 tablet_copy_client.cc:538] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:40.843164 32064 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:40.864326 32064 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:40.866493 32064 tablet_bootstrap.cc:492] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:40.867250 32064 ts_tablet_manager.cc:1403] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.024s	user 0.022s	sys 0.001s
I20260709 12:23:40.870254 32064 raft_consensus.cc:359] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:40.871201 32064 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:40.871963 32064 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:40.881968 32064 ts_tablet_manager.cc:1434] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.007s
I20260709 12:23:40.883747 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-45809554a739456ea0a596484ede5ac7 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:40.884176 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-45809554a739456ea0a596484ede5ac7 on tablet 45809554a739456ea0a596484ede5ac7 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:40.911607 32064 ts_tablet_manager.cc:933] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:40.935238 32070 ts_tablet_manager.cc:933] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:40.940569 32070 tablet_copy_client.cc:323] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:40.942147 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet b1c1f8d5241540fdb4fb4daa1691260f from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:40.942720 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet b1c1f8d5241540fdb4fb4daa1691260f from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-b1c1f8d5241540fdb4fb4daa1691260f
I20260709 12:23:40.975878 31002 tablet_copy_source_session.cc:215] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:40.982721 32070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c1f8d5241540fdb4fb4daa1691260f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:40.983672 32064 tablet_copy_client.cc:323] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:40.987421 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet ba8a01bf2a514da2805eb75a7c28c412 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:40.987947 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet ba8a01bf2a514da2805eb75a7c28c412 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-ba8a01bf2a514da2805eb75a7c28c412
I20260709 12:23:40.995265 31002 tablet_copy_source_session.cc:215] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:40.999603 32064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba8a01bf2a514da2805eb75a7c28c412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.009917 32070 tablet_copy_client.cc:806] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:41.010623 32070 tablet_copy_client.cc:670] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:41.019217 32072 ts_tablet_manager.cc:933] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:41.019586 32064 tablet_copy_client.cc:806] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:41.020311 32064 tablet_copy_client.cc:670] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:41.022456 32072 tablet_copy_client.cc:323] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:41.026294 32070 tablet_copy_client.cc:538] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:41.039382 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 635264f51b2d4410a1da9b76606a6350 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:41.039958 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 635264f51b2d4410a1da9b76606a6350 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-635264f51b2d4410a1da9b76606a6350
I20260709 12:23:41.053580 32064 tablet_copy_client.cc:538] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:41.066380 32064 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:41.076382 32070 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:41.082289 31154 tablet_copy_source_session.cc:215] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:41.117522 32070 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:41.121470 32072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635264f51b2d4410a1da9b76606a6350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.118878 32070 tablet_bootstrap.cc:492] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:41.144770 32070 ts_tablet_manager.cc:1403] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.069s	user 0.019s	sys 0.006s
I20260709 12:23:41.152089 32064 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:41.153024 32064 tablet_bootstrap.cc:492] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:41.153841 32064 ts_tablet_manager.cc:1403] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.088s	user 0.032s	sys 0.022s
I20260709 12:23:41.160841 32064 raft_consensus.cc:359] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:41.161712 32064 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:41.162426 32064 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:41.164772 32064 ts_tablet_manager.cc:1434] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:23:41.171772 32070 raft_consensus.cc:359] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:41.172665 32070 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:41.173355 32070 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:41.175928 32070 ts_tablet_manager.cc:1434] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.031s	user 0.002s	sys 0.003s
I20260709 12:23:41.188594 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-ba8a01bf2a514da2805eb75a7c28c412 received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:41.189040 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-ba8a01bf2a514da2805eb75a7c28c412 on tablet ba8a01bf2a514da2805eb75a7c28c412 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:41.642755 32072 tablet_copy_client.cc:806] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:41.643437 32072 tablet_copy_client.cc:670] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:41.700246 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-b1c1f8d5241540fdb4fb4daa1691260f received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:41.700698 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-b1c1f8d5241540fdb4fb4daa1691260f on tablet b1c1f8d5241540fdb4fb4daa1691260f with peer a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:23:41.735355 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 62242.2us
I20260709 12:23:41.744683 31696 raft_consensus.cc:1217] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:41.752202 31694 raft_consensus.cc:1217] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:41.772904 32072 tablet_copy_client.cc:538] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:41.779026 31695 raft_consensus.cc:1217] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:41.783978 32072 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:41.876293 32072 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:41.877372 32072 tablet_bootstrap.cc:492] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:41.878159 32072 ts_tablet_manager.cc:1403] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.094s	user 0.021s	sys 0.010s
I20260709 12:23:41.881227 32072 raft_consensus.cc:359] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:41.882115 32072 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:41.882874 32072 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:41.893772 32072 ts_tablet_manager.cc:1434] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:23:41.903070 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-635264f51b2d4410a1da9b76606a6350 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:41.903502 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-635264f51b2d4410a1da9b76606a6350 on tablet 635264f51b2d4410a1da9b76606a6350 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:42.012661 32081 raft_consensus.cc:1064] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:42.015357 32081 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:42.023455 31695 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.025813 31136 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.026193 31064 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.027521 32058 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.029207 32034 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.030916 31976 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.065630 32034 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.076174 31135 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.079604 31694 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.082702 31065 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.091421 30885 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.127292 31698 raft_consensus.cc:1217] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:42.175015 30988 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:42.206902 31695 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:42.208076 31065 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:42.208406 32080 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:42.209607 32034 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:42.230780 31064 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.232740 32080 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.239648 31694 raft_consensus.cc:2955] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.293218 32058 raft_consensus.cc:1064] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:42.295800 32058 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:42.313944 31058 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.336216 32077 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.351491 31698 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:42.360957 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 109366us
I20260709 12:23:42.374747 30887 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.381223 32081 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:42.395408 31136 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.401232 31976 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:23:42.407821 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 117826us
I20260709 12:23:42.421546 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b1c1f8d5241540fdb4fb4daa1691260f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:42.422973 32080 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.428798 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet b1c1f8d5241540fdb4fb4daa1691260f with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:42.430083 31065 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.440599 30887 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.447701 32038 raft_consensus.cc:1064] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:42.450662 32038 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:42.432659 31694 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.474424 31135 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.476679 31698 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.478766 32082 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.480377 30988 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.481019 32082 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:42.489841 32110 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:42.499657 31136 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.500694 32108 tablet_replica.cc:333] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:42.502200 32108 raft_consensus.cc:2243] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:42.505153 32108 raft_consensus.cc:2272] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:42.512610 32108 ts_tablet_manager.cc:1916] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:42.524096 32110 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.527683 31136 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.531252 32108 ts_tablet_manager.cc:1929] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:42.531764 32108 log.cc:1199] T b1c1f8d5241540fdb4fb4daa1691260f P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/b1c1f8d5241540fdb4fb4daa1691260f
I20260709 12:23:42.533789 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet b1c1f8d5241540fdb4fb4daa1691260f (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:42.549891 31698 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.552681 30992 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.561156 30886 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.583300 31065 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:42.591006 31698 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:42.592303 31136 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:42.594025 32082 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:42.603199 32110 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:42.607270 32107 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.610687 31698 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.609601 31135 raft_consensus.cc:2955] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:23:42.644707 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080011d0e0 after lost signal to thread 29848
W20260709 12:23:42.645666 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080011d7c0 after lost signal to thread 30924
W20260709 12:23:42.646755 31013 debug-util.cc:398] Leaking SignalData structure 0x7b0800280d60 after lost signal to thread 31016
W20260709 12:23:42.647735 31013 debug-util.cc:398] Leaking SignalData structure 0x7b0800324620 after lost signal to thread 31093
W20260709 12:23:42.648720 31013 debug-util.cc:398] Leaking SignalData structure 0x7b08003302c0 after lost signal to thread 31168
W20260709 12:23:42.649739 31013 debug-util.cc:398] Leaking SignalData structure 0x7b0800303760 after lost signal to thread 31722
I20260709 12:23:42.664011 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 45809554a739456ea0a596484ede5ac7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:42.676999 30886 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:42.695495 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 45809554a739456ea0a596484ede5ac7 with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:42.700749 32088 raft_consensus.cc:1064] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:42.710644 32088 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:42.730581 32115 tablet_replica.cc:333] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:42.733125 31065 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.735351 31698 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.737470 30985 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.740080 32104 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:42.741918 32104 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:42.746982 32103 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:42.771584 31065 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.768882 32088 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.813972 30886 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.848969 31698 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.895363 30988 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:42.939683 32115 raft_consensus.cc:2243] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:42.940435 32115 raft_consensus.cc:2272] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:42.944386 32115 ts_tablet_manager.cc:1916] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:42.987968 32115 ts_tablet_manager.cc:1929] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:42.988462 32115 log.cc:1199] T 45809554a739456ea0a596484ede5ac7 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/45809554a739456ea0a596484ede5ac7
I20260709 12:23:42.990073 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 45809554a739456ea0a596484ede5ac7 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:43.002774 31135 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:43.013900 31695 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.014201 30988 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.023603 32088 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:43.025440 32088 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:43.034049 32032 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:43.077855 30985 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:43.088128 31695 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.088128 31135 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.091023 32080 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:43.091670 31976 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:43.105571 32080 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:43.116499 31695 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:43.123418 31698 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:43.130427 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ba8a01bf2a514da2805eb75a7c28c412 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.140635 30887 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:43.142247 30988 raft_consensus.cc:2955] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:43.154701 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet 635264f51b2d4410a1da9b76606a6350 with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:43.204628 30887 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:43.208458 31141 raft_consensus.cc:2955] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:43.224908 31052 tablet_service.cc:1558] Processing DeleteTablet for tablet ba8a01bf2a514da2805eb75a7c28c412 with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:43.232262 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 635264f51b2d4410a1da9b76606a6350 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.258474 32120 tablet_replica.cc:333] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:43.259581 32120 raft_consensus.cc:2243] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.262815 32120 raft_consensus.cc:2272] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:43.273350 32120 ts_tablet_manager.cc:1916] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:43.293598 32120 ts_tablet_manager.cc:1929] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:43.294005 32120 log.cc:1199] T 635264f51b2d4410a1da9b76606a6350 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/635264f51b2d4410a1da9b76606a6350
I20260709 12:23:43.295718 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet 635264f51b2d4410a1da9b76606a6350 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:43.297367 32120 tablet_replica.cc:333] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:43.298511 32120 raft_consensus.cc:2243] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.301458 32120 raft_consensus.cc:2272] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:43.308064 32120 ts_tablet_manager.cc:1916] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:43.332660 32120 ts_tablet_manager.cc:1929] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:43.333300 32120 log.cc:1199] T ba8a01bf2a514da2805eb75a7c28c412 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/ba8a01bf2a514da2805eb75a7c28c412
I20260709 12:23:43.334860 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet ba8a01bf2a514da2805eb75a7c28c412 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:44.358139 31065 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:44.382844 31135 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.386812 32107 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.393847 30988 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:44.393189 31033 consensus_peers.cc:597] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ec17bf7d52c4cb69f4d488858fecc8b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:44.397571 31989 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.406306 32110 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.415465 30988 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.420639 31065 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:44.427474 30887 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.448422 31141 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.453456 31135 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:44.455010 31033 consensus_peers.cc:597] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d522233ee4c4536b4fc944d7866c4d9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:44.455654 31989 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.469578 30992 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.471730 32110 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.491595 31989 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.530737 31141 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.556465 30985 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.562623 30887 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.565532 31141 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:44.599596 31065 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.603844 32122 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.618561 30985 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:44.620671 31109 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 044e61a321dd4521be7f9ee796ca44dc. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:44.623942 32103 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:44.754856 32139 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:23:44.799048 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47609us
W20260709 12:23:44.803274 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44673.3us
I20260709 12:23:44.830150 30988 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:44.833083 30987 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.820808 31064 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:44.883342 30889 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:44.913076 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 91383.8us
W20260709 12:23:44.958652 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 92660.2us
W20260709 12:23:44.962795 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 68059us
I20260709 12:23:44.977435 31058 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.983880 31136 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.984426 32080 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:44.985555 30958 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 363b01e660fa4b36aa4a5112bab1ac09. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:44.997820 32145 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:23:45.005455 32150 ts_tablet_manager.cc:933] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:45.007208 32149 ts_tablet_manager.cc:933] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:45.035646 31976 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:45.039521 31058 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:45.071277 32149 tablet_copy_client.cc:323] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:45.073693 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 0ec17bf7d52c4cb69f4d488858fecc8b from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:45.074352 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 0ec17bf7d52c4cb69f4d488858fecc8b from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-0ec17bf7d52c4cb69f4d488858fecc8b
I20260709 12:23:45.086133 31142 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:45.094141 30889 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:45.109725 32153 ts_tablet_manager.cc:933] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:45.119990 31078 tablet_copy_source_session.cc:215] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.123904 32149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ec17bf7d52c4cb69f4d488858fecc8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.131067 32150 tablet_copy_client.cc:323] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:45.133885 32153 tablet_copy_client.cc:323] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:45.133529 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 4d522233ee4c4536b4fc944d7866c4d9 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:45.134665 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 4d522233ee4c4536b4fc944d7866c4d9 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-4d522233ee4c4536b4fc944d7866c4d9
I20260709 12:23:45.135931 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 044e61a321dd4521be7f9ee796ca44dc from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:45.136415 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 044e61a321dd4521be7f9ee796ca44dc from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-044e61a321dd4521be7f9ee796ca44dc
I20260709 12:23:45.143038 31154 tablet_copy_source_session.cc:215] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.146369 32153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044e61a321dd4521be7f9ee796ca44dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.153867 31078 tablet_copy_source_session.cc:215] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.163601 32149 tablet_copy_client.cc:806] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.164242 32153 tablet_copy_client.cc:806] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.164330 32149 tablet_copy_client.cc:670] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.164564 32150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d522233ee4c4536b4fc944d7866c4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.164909 32153 tablet_copy_client.cc:670] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.168716 32153 tablet_copy_client.cc:538] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.180363 32153 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:45.190244 32149 tablet_copy_client.cc:538] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.218580 32153 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:45.219663 32153 tablet_bootstrap.cc:492] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:45.220402 32153 ts_tablet_manager.cc:1403] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.018s
I20260709 12:23:45.223690 32153 raft_consensus.cc:359] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:45.225478 32153 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:45.226260 32153 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:45.232833 32149 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:45.229472 32150 tablet_copy_client.cc:806] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.235567 32150 tablet_copy_client.cc:670] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.239220 32153 ts_tablet_manager.cc:1434] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.008s
I20260709 12:23:45.243288 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-044e61a321dd4521be7f9ee796ca44dc received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:45.243752 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-044e61a321dd4521be7f9ee796ca44dc on tablet 044e61a321dd4521be7f9ee796ca44dc with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:45.246915 32150 tablet_copy_client.cc:538] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.260207 32150 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:45.308591 32149 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:45.309580 32149 tablet_bootstrap.cc:492] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:45.310379 32149 ts_tablet_manager.cc:1403] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.081s	user 0.015s	sys 0.015s
I20260709 12:23:45.313779 32149 raft_consensus.cc:359] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:45.326128 32149 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:45.326890 32149 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:45.327711 32150 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:45.328627 32150 tablet_bootstrap.cc:492] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:45.329777 32149 ts_tablet_manager.cc:1434] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:23:45.331746 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-0ec17bf7d52c4cb69f4d488858fecc8b received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:45.332185 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-0ec17bf7d52c4cb69f4d488858fecc8b on tablet 0ec17bf7d52c4cb69f4d488858fecc8b with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:45.333225 32150 ts_tablet_manager.cc:1403] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.014s
I20260709 12:23:45.336500 32150 raft_consensus.cc:359] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:45.337409 32150 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:45.338165 32150 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:45.341199 32150 ts_tablet_manager.cc:1434] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:23:45.343220 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-4d522233ee4c4536b4fc944d7866c4d9 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:45.343612 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-4d522233ee4c4536b4fc944d7866c4d9 on tablet 4d522233ee4c4536b4fc944d7866c4d9 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:45.371629 31698 raft_consensus.cc:1217] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:45.381369 32150 ts_tablet_manager.cc:933] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:45.402441 32150 tablet_copy_client.cc:323] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:45.409235 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet 363b01e660fa4b36aa4a5112bab1ac09 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:45.409832 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet 363b01e660fa4b36aa4a5112bab1ac09 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-363b01e660fa4b36aa4a5112bab1ac09
I20260709 12:23:45.425022 31002 tablet_copy_source_session.cc:215] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.430639 32150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363b01e660fa4b36aa4a5112bab1ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.443943 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:45.461949 32150 tablet_copy_client.cc:806] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.462572 32150 tablet_copy_client.cc:670] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.466455 32150 tablet_copy_client.cc:538] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.478547 31698 raft_consensus.cc:1217] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:45.479565 32150 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:45.532311 32150 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:45.533365 32150 tablet_bootstrap.cc:492] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:45.534042 32150 ts_tablet_manager.cc:1403] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.007s
I20260709 12:23:45.537037 32150 raft_consensus.cc:359] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:45.537878 32150 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:45.538569 32150 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:45.545305 32150 ts_tablet_manager.cc:1434] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.002s
I20260709 12:23:45.547036 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-363b01e660fa4b36aa4a5112bab1ac09 received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:45.547475 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-363b01e660fa4b36aa4a5112bab1ac09 on tablet 363b01e660fa4b36aa4a5112bab1ac09 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:45.635215 31694 raft_consensus.cc:1217] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:45.675072 31058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:45.675875 31058 raft_consensus.cc:606] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:45.678753 31065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45809554a739456ea0a596484ede5ac7"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:45.679257 31065 raft_consensus.cc:606] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:45.681830 31065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:45.682317 31065 raft_consensus.cc:606] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:45.692674 32110 raft_consensus.cc:993] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:45.693075 32110 raft_consensus.cc:1081] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:45.694739 31698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45809554a739456ea0a596484ede5ac7"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51374
I20260709 12:23:45.695358 31698 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:45.695706 31698 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.702839 31698 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:45.706650 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:45.707247 31141 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.713184 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:45.713804 31058 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:45.714092 31058 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:45.721019 31141 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:45.719216 31058 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:45.722404 31058 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.722378 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:45.723701 32158 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:45.730706 31058 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:45.745086 30985 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76399531471a4790a56d52a6ea052864"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:45.745627 30985 raft_consensus.cc:606] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:45.745906 31698 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:45.749836 31058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:45.750488 31058 raft_consensus.cc:606] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:45.777796 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:45.778039 32158 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:45.778360 31142 raft_consensus.cc:606] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:45.781363 31058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:45.779022 32158 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:45.782051 31058 raft_consensus.cc:606] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:45.786829 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:23:45.788401 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:45.789364 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:45.796324 32080 raft_consensus.cc:993] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:45.796774 32080 raft_consensus.cc:1081] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:45.801009 31058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76399531471a4790a56d52a6ea052864"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:23:45.801549 31058 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:45.801918 31058 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:45.814481 31058 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:45.817123 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:45.817711 30985 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:45.818003 30985 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:45.818732 30985 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:45.819859 30985 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:45.837697 30887 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:45.848645 32143 raft_consensus.cc:993] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:45.849135 32110 raft_consensus.cc:1081] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:45.850829 30987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:45.851295 30987 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:45.851620 30987 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.857661 30987 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:45.860399 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:45.860896 31060 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:45.861174 31060 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:45.861363 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:45.862077 31142 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:45.861879 31060 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:45.862974 31060 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.869017 31060 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:45.871904 30955 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:45.872870 32145 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:45.873346 32145 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:45.874306 32145 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:45.891697 31058 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:45.897264 30985 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:45.899850 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:45.910290 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:45.910769 31141 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.920994 32143 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:45.921489 32143 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:45.937703 32143 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:45.945618 31141 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:45.907214 32110 raft_consensus.cc:1064] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:45.959501 32110 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:45.961946 30889 catalog_manager.cc:5697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.900434 31142 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:45.979728 31698 raft_consensus.cc:1217] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:45.898288 30987 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:46.028940 31142 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.031739 31694 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.033876 32143 raft_consensus.cc:993] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:46.034369 30987 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.035732 32137 raft_consensus.cc:1064] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:46.034375 32143 raft_consensus.cc:1081] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:46.048673 31696 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51374
I20260709 12:23:46.054950 31696 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.055305 31696 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.048840 32143 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:23:46.047274 32137 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:46.064271 32107 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.065152 32134 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.087704 30886 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.089908 32134 raft_consensus.cc:993] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:46.090596 32177 raft_consensus.cc:1081] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:46.096802 32143 raft_consensus.cc:1064] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:46.097625 30985 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:46.099221 30985 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.100149 32107 raft_consensus.cc:993] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:46.099632 30985 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.100078 32143 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:46.100690 32107 raft_consensus.cc:1081] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:46.103168 31695 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51374
I20260709 12:23:46.103688 31695 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.104038 31695 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.083966 31696 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:46.118595 30985 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:46.126428 30985 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:46.127028 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:46.127183 31058 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.127616 31060 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.127925 31060 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:46.130481 31696 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:46.143550 32122 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.145408 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:46.145911 31058 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.146165 31058 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:46.146832 31058 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:46.150600 31060 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:46.151777 31060 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.153232 31058 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.176783 31695 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:46.178514 32172 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.180138 31695 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:46.184880 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:46.185386 30987 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.186300 31060 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:46.186429 31058 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:46.192284 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:46.192879 31063 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.193156 31063 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:46.194423 31064 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:46.193246 30986 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.195677 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:46.196194 31141 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.197714 30985 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.198709 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:46.194006 31063 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:46.216033 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:46.216187 30987 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:46.216572 31142 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.218361 31063 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.220816 32080 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.222680 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:46.226681 31695 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.223280 32122 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:23:46.247009 32167 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.241875 31698 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.250026 31141 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:46.252306 31142 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:46.252388 31063 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:46.219738 32158 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:23:46.263886 32182 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:46.298564 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57330.4us
I20260709 12:23:46.303341 32080 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:46.308070 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:46.322206 31698 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.304875 32183 raft_consensus.cc:993] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:46.328841 31140 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:46.329674 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 108397us
I20260709 12:23:46.329946 32080 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:46.311620 32167 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:46.321822 32158 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.333792 32183 raft_consensus.cc:1081] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:46.338891 31140 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:46.339802 31063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:46.340318 31063 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.340669 31063 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.340806 32189 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.348594 31063 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:46.351300 32158 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:46.354031 31063 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:46.355237 32177 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:23:46.359403 32158 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:23:46.393043 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 47389.2us
I20260709 12:23:46.405894 32192 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:23:46.407595 32167 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:23:46.538534 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 142091us
I20260709 12:23:46.538731 31063 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.542637 32166 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.561861 30886 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:46.575078 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 240029us
I20260709 12:23:46.577945 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:46.578511 31140 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.578792 31140 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:46.579444 31140 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:46.579479 31136 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.580646 31140 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.583922 31137 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:46.594449 31140 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:46.608453 30887 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.615545 31058 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:46.627673 32211 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.645248 32190 raft_consensus.cc:1064] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:46.652536 30887 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:46.661049 32190 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:46.666425 30887 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:46.676764 31058 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.684119 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:46.684741 30991 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.680965 30986 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.688984 31144 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:46.689205 31698 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:46.690244 30983 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.700659 32189 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.702030 32203 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:46.708007 31144 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.713550 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:46.715101 32200 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.715605 32200 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:46.718609 30991 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:46.716636 32200 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:46.742965 32080 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:46.747853 30983 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:46.739122 32143 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:23:46.775597 30885 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:23:46.781857 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 81626us
I20260709 12:23:46.793279 32189 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:23:46.811544 32195 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:23:46.847781 31058 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:46.849673 32167 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:46.834816 32134 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.869931 32189 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.881667 31694 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.885279 31141 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.894639 31136 raft_consensus.cc:2955] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:46.910159 31142 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:46.912868 31063 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.914975 32166 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:46.918089 32167 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:46.921106 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 193808us
I20260709 12:23:46.944852 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4d522233ee4c4536b4fc944d7866c4d9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:46.996971 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:46.998915 31065 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:47.015416 32167 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:47.020284 31698 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.026535 31139 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.025427 31698 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.030830 31144 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:47.037781 32201 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:23:47.076293 30885 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.099161 31697 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.092983 32187 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
W20260709 12:23:47.165351 32228 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:23:47.172241 30990 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:47.199095 31058 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:47.228101 32166 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:47.197721 32167 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 12:23:47.239357 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54312.6us
I20260709 12:23:47.266476 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d522233ee4c4536b4fc944d7866c4d9 with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:47.336928 31142 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:47.292716 30983 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:47.343780 32195 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:23:47.336113 32201 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 12:23:47.398626 32238 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:23:47.464658 31139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:47.465654 31139 raft_consensus.cc:606] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:47.467422 32240 tablet_replica.cc:333] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:47.489166 32240 raft_consensus.cc:2243] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:47.494623 32240 raft_consensus.cc:2272] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:47.548702 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 210485us
I20260709 12:23:47.550632 30991 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.567391 32110 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:23:47.566867 32240 ts_tablet_manager.cc:1916] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:47.617955 31139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:47.618574 31139 raft_consensus.cc:606] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:23:47.640707 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 72540.3us
I20260709 12:23:47.657361 31064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:47.657949 31064 raft_consensus.cc:606] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:47.649616 30886 catalog_manager.cc:5697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.673434 31142 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.677950 32184 raft_consensus.cc:993] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:47.678473 32242 raft_consensus.cc:993] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:47.678936 32242 raft_consensus.cc:1081] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:47.679286 32183 raft_consensus.cc:1081] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
W20260709 12:23:47.680631 30876 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leader transfer in progress [suppressed 2 similar messages]
I20260709 12:23:47.678625 31694 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.685930 31697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51360
I20260709 12:23:47.686637 31697 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.687036 31697 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.658012 32201 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.691569 31064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:47.692095 31064 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.692451 31064 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.700062 31067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:47.700706 31067 raft_consensus.cc:606] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:47.704491 31139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:47.705022 31139 raft_consensus.cc:606] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:23:47.731464 30876 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 50962.1us
I20260709 12:23:47.736285 31064 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:47.742959 31697 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:47.754916 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:47.754145 31067 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.762562 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:47.763135 30992 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.763912 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:47.764506 31142 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.764846 31142 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:47.765697 31142 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:47.766633 32240 ts_tablet_manager.cc:1929] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:47.767062 31142 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.767385 31697 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:47.767797 31064 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:23:47.758659 30876 catalog_manager.cc:4755] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): ChangeConfig:REMOVE_PEER RPC failed for tablet 044e61a321dd4521be7f9ee796ca44dc: Remote error: Service unavailable: leader transfer in progress
I20260709 12:23:47.769496 30990 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.772567 32240 log.cc:1199] T 4d522233ee4c4536b4fc944d7866c4d9 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/4d522233ee4c4536b4fc944d7866c4d9
I20260709 12:23:47.786408 30983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:47.787413 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 4d522233ee4c4536b4fc944d7866c4d9 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:47.787562 30983 raft_consensus.cc:606] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:47.789000 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:23:47.790375 30992 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:47.790521 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:47.791527 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:47.783918 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:47.804709 31058 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.784770 30990 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:47.776930 31142 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:47.812222 32193 raft_consensus.cc:993] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:47.812389 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:47.812695 32193 raft_consensus.cc:1081] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:47.814659 31058 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:47.817037 32158 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.817576 32158 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:47.818852 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:47.819371 31142 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.819656 31142 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:47.818470 32158 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:47.822374 31142 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:47.823556 31142 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.824807 31058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:47.825318 31058 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.825574 31058 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.831043 31058 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:47.832557 31058 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:47.839512 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:47.840430 32134 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.842582 31142 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:47.842900 32134 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:47.844071 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:47.843838 32134 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:47.844604 30992 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.845146 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:47.845672 31136 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.845958 31136 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:47.846668 31136 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:47.847839 31136 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.850075 30992 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
W20260709 12:23:47.851312 30876 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 044e61a321dd4521be7f9ee796ca44dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ce27a72886964e4287059a1d66bd5b6b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.856537 31136 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:47.859690 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:47.864481 32200 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.864964 32200 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:47.865859 32200 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:47.895581 30887 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.936170 30887 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.947309 30887 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:47.992147 30983 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:47.999845 31694 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:48.007417 31058 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:48.012060 32145 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:48.021612 32080 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.022624 32200 raft_consensus.cc:993] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:48.026849 32200 raft_consensus.cc:1081] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:48.029395 31694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51374
I20260709 12:23:48.030000 31694 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.029327 31064 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.030390 31694 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.991963 30886 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.037421 31694 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:48.037218 30886 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.040385 32187 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:48.045964 31695 raft_consensus.cc:2955] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.053829 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:48.054400 31058 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.054690 31058 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:48.055435 31058 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:48.057004 31058 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.080215 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:48.080883 31142 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.082885 31694 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:48.091522 31058 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:48.100521 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 363b01e660fa4b36aa4a5112bab1ac09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:48.106739 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:23:48.108384 32158 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.108847 32158 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:48.109651 32158 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:23:48.128708 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 52357.5us
I20260709 12:23:48.149122 32190 raft_consensus.cc:993] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:48.149595 32190 raft_consensus.cc:1081] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:48.164322 30887 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.166373 31058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:23:48.166872 31058 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.167444 31058 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:48.179728 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 363b01e660fa4b36aa4a5112bab1ac09 with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:48.187480 30889 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:48.215378 31058 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:48.219723 32263 tablet_replica.cc:333] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:48.224280 31136 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:48.242434 32195 raft_consensus.cc:993] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:48.248953 31058 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:48.243335 31064 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:23:48.246147 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 80123.5us
I20260709 12:23:48.248802 32263 raft_consensus.cc:2243] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:48.251260 32263 raft_consensus.cc:2272] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:48.161650 31142 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:48.253386 32198 raft_consensus.cc:1081] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:48.254976 32158 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:23:48.283214 32263 ts_tablet_manager.cc:1916] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:48.290051 30992 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:48.291566 32158 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:48.332885 32263 ts_tablet_manager.cc:1929] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:48.333482 32263 log.cc:1199] T 363b01e660fa4b36aa4a5112bab1ac09 P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/363b01e660fa4b36aa4a5112bab1ac09
I20260709 12:23:48.335271 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 363b01e660fa4b36aa4a5112bab1ac09 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
W20260709 12:23:48.343915 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 89491.2us
I20260709 12:23:48.350939 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:48.351464 30991 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:48.351740 30991 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:48.352442 30991 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:48.353610 30991 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:48.372397 30991 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:48.383491 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:48.384227 31142 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:48.390455 31139 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.392462 31695 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:48.392702 30984 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.394177 31694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51374
I20260709 12:23:48.394907 31694 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.395342 31694 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.399742 31139 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:48.414677 31064 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:48.416014 32188 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:48.421833 31694 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:48.425030 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:48.425346 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:48.425745 31067 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.425987 30984 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.426082 31067 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:48.427074 31067 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:48.428556 31067 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.429693 31142 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:48.444721 31067 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:48.454358 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:23:48.455876 32158 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.457556 32158 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:48.458729 32158 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:48.445195 31694 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:48.434190 30984 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:48.441984 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:48.463677 32243 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:48.464200 32243 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:48.464996 32243 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:48.471410 32134 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:23:48.486078 30886 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:48.498386 31136 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.505793 32134 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:48.508839 31067 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.514148 30992 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.517899 32167 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.520207 32264 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.521433 32203 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:48.525486 32203 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.531458 30886 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.555096 32243 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.578756 31141 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.586601 32188 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.654914 31694 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:48.662637 32200 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.670894 30992 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.671561 31136 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.672412 32167 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.673880 32167 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.674557 31694 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.681605 32188 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.717903 30986 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.730470 31139 raft_consensus.cc:2955] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.686015 31136 raft_consensus.cc:2955] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.761942 30886 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.765285 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0ec17bf7d52c4cb69f4d488858fecc8b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:48.790670 30885 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 3 to 5, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:48.791366 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 044e61a321dd4521be7f9ee796ca44dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20260709 12:23:48.814527 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet 044e61a321dd4521be7f9ee796ca44dc with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:48.823029 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ec17bf7d52c4cb69f4d488858fecc8b with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:48.839048 32286 tablet_replica.cc:333] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:48.840111 32286 raft_consensus.cc:2243] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:48.842162 32285 tablet_replica.cc:333] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:48.854948 32286 raft_consensus.cc:2272] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:48.855439 32285 raft_consensus.cc:2243] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:48.856078 32285 raft_consensus.cc:2272] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:48.858170 32286 ts_tablet_manager.cc:1916] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:48.859347 32285 ts_tablet_manager.cc:1916] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:48.865959 31136 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:48.867868 32264 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:48.888814 32286 ts_tablet_manager.cc:1929] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:48.889290 32286 log.cc:1199] T 0ec17bf7d52c4cb69f4d488858fecc8b P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/0ec17bf7d52c4cb69f4d488858fecc8b
I20260709 12:23:48.892282 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 0ec17bf7d52c4cb69f4d488858fecc8b (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:48.902645 30992 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:48.904352 32243 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:48.916011 32285 ts_tablet_manager.cc:1929] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:48.916553 32285 log.cc:1199] T 044e61a321dd4521be7f9ee796ca44dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/044e61a321dd4521be7f9ee796ca44dc
I20260709 12:23:48.918241 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet 044e61a321dd4521be7f9ee796ca44dc (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:48.976056 30992 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.977623 32167 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:48.986897 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:48.995610 31067 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:48.999210 32188 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.076207 31067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:49.076807 31067 raft_consensus.cc:606] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:49.080552 31139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48047b2b139146c38a821328cf7bd148"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:49.081115 31139 raft_consensus.cc:606] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:49.091459 31064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:49.092003 31064 raft_consensus.cc:606] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:49.094749 31064 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:49.095230 31064 raft_consensus.cc:606] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:49.098388 30992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:49.098904 30992 raft_consensus.cc:606] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:49.101269 31139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:49.101694 31139 raft_consensus.cc:606] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:49.102741 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:23:49.103817 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:49.104524 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:49.129220 32264 raft_consensus.cc:993] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:49.129768 32264 raft_consensus.cc:1081] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:49.131314 30992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:49.131796 30992 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.132115 30992 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.138733 30992 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.142202 32243 raft_consensus.cc:993] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:49.142709 32243 raft_consensus.cc:1081] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:49.144397 31695 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51374
I20260709 12:23:49.144937 31695 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.145347 31695 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.148267 30992 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:49.149327 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:49.149916 31064 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.150000 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:49.150286 31064 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:49.150641 31139 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.151126 31064 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.152364 31064 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.154009 31695 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:49.155827 31695 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:49.156749 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:49.156783 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:49.157526 31136 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.157408 31067 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.158754 31067 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:49.159442 31067 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:49.162499 31067 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.165439 31136 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:49.165762 31139 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:49.166764 31064 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:49.167116 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:49.167295 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:49.167846 32236 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.169062 32284 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.172031 32284 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:49.172858 32284 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:49.183812 30889 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:49.228886 31067 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:49.232357 32236 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:49.233454 32236 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.252276 30889 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.261861 32292 raft_consensus.cc:993] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:49.262356 32292 raft_consensus.cc:1081] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:49.264863 31694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51362
I20260709 12:23:49.265340 31694 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.265687 31694 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.274150 31694 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:49.276604 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:49.276770 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:49.277170 30992 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.277417 31067 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.277531 30992 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:49.278401 30992 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:49.279450 30992 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.281469 31694 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:49.283891 31067 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:49.289081 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:23:49.290035 32167 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.290598 32167 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:49.291520 32167 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:49.294098 30992 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:49.334779 30889 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:49.455478 32264 raft_consensus.cc:993] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:49.455976 32264 raft_consensus.cc:1081] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:49.460047 31139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45360
I20260709 12:23:49.460561 31139 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.460901 31139 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.467175 31139 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.470166 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:49.470958 30992 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.472496 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:49.473026 31064 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.473086 31139 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:49.473397 31064 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:49.474102 31064 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.475817 31064 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.480868 30992 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:49.481969 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:49.482795 32242 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.483234 32242 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:49.483989 32242 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.486783 31064 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:49.506369 30889 catalog_manager.cc:5697] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:49.512431 32184 raft_consensus.cc:993] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:49.512868 32184 raft_consensus.cc:1081] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:49.515942 31058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48047b2b139146c38a821328cf7bd148"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:49.516474 31058 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.516812 31058 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.523866 31058 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.526762 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:49.527307 30992 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.539951 30992 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:49.541011 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:49.542471 32243 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.543011 32243 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:49.543874 32243 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.549194 31058 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:49.556931 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:49.559012 31139 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.559386 31139 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:49.559037 30889 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.560361 31139 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.561586 31139 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.577491 31139 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:49.593945 32184 raft_consensus.cc:993] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:49.595283 32184 raft_consensus.cc:1081] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:49.597061 31067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:49.597558 31067 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.597880 31067 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.613243 31067 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.615566 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:49.616089 30992 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.618901 31067 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:49.624010 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:49.625448 31139 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.625799 31139 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:49.624962 30992 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:49.627068 31139 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.627754 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:49.628466 31139 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.631800 32243 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.633519 32243 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:49.634721 32243 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:49.649489 31139 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:49.663540 30889 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.675534 31067 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.676887 32167 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.683136 31139 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.685709 30992 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.687579 32167 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.700861 32284 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.719951 31136 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.722760 32291 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.787243 31058 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.789678 31058 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:49.791010 32284 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.794612 32292 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:49.953811 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 66973.1us
I20260709 12:23:49.993794 30992 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.001395 32242 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:50.055203 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 160481us
I20260709 12:23:50.056959 31058 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.068858 32312 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:50.050501 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 148215us
I20260709 12:23:50.104861 31136 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.108296 31136 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.122849 32200 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.125654 32200 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.165799 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:50.187889 30991 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.187711 30992 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:50.190663 30986 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.192600 32316 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.214479 32316 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.217742 31067 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:50.219131 32315 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.237685 31136 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:50.274680 30958 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 83f49f3c3cc44994877e69c3ff83090e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:50.289690 30992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0afdb545bb8f485993259007a00c1841"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:50.290292 30992 raft_consensus.cc:606] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:50.293754 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "205fb3f3a3844073a49da13542da2989"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:50.294412 31060 raft_consensus.cc:606] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:50.301558 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:50.302141 31136 raft_consensus.cc:606] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:50.305050 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:50.305651 31060 raft_consensus.cc:606] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:50.306907 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:50.308686 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:50.309418 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:50.313956 32320 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.343712 31067 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.341331 32320 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.350878 32292 raft_consensus.cc:993] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:50.351387 32292 raft_consensus.cc:1081] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:50.355073 32319 raft_consensus.cc:993] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:50.355564 32319 raft_consensus.cc:1081] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:50.359315 31136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "205fb3f3a3844073a49da13542da2989"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45360
I20260709 12:23:50.359959 31136 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.360288 31136 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.367431 31136 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:50.370168 31694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0afdb545bb8f485993259007a00c1841"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51362
I20260709 12:23:50.371634 31694 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.371974 31694 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.383772 31139 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.384627 31694 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:50.387239 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:50.387715 31136 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:50.388726 31694 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:50.389931 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:50.390537 31058 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.387749 30986 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.394919 30986 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:50.395709 30986 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:50.396948 30986 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.399286 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:50.399859 30992 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.421798 30885 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.426764 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:50.433590 31060 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.434275 31060 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:50.435077 31060 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:50.450687 31058 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:50.450726 30992 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:50.453328 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:23:50.453619 31060 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.454433 32188 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.454882 32188 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:50.456878 30986 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:50.457041 32188 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:50.462041 31109 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:50.463037 32184 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.463553 32184 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:50.464401 32184 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:50.468211 31060 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:50.493412 31136 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:50.518455 31067 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:50.521860 32184 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.525811 30886 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.551342 30885 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:50.553727 31109 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f57ce098bf541c4b1ca32a9c4b8cec0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:50.577188 30986 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:50.578653 32184 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.586427 32312 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.597343 31060 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.607180 31064 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:50.622197 32184 raft_consensus.cc:993] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:50.622682 32184 raft_consensus.cc:1081] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:50.624476 30986 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.627384 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:50.627880 31060 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.628242 31060 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.632844 31136 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:50.636588 31060 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:50.637512 30992 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:50.638386 31060 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:23:50.639693 32316 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.642198 32316 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.644362 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:23:50.644441 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:50.645052 31694 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.646467 30885 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:23:50.651804 31033 consensus_peers.cc:597] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 05394269d1a4461bb79d6f93ab25ff4e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:50.665266 31136 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:50.666966 31136 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:50.667922 31136 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:50.684659 31694 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:50.686286 31136 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:50.750552 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:23:50.755897 32311 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:50.756721 32311 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:50.757932 32311 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:50.766846 32343 ts_tablet_manager.cc:933] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:50.776933 32319 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.778620 31136 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:50.817001 30983 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.840636 31060 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:50.851550 30887 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.852772 32316 raft_consensus.cc:993] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:50.853287 32200 raft_consensus.cc:1081] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:50.854933 32343 tablet_copy_client.cc:323] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:50.856676 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet 83f49f3c3cc44994877e69c3ff83090e from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:50.857271 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet 83f49f3c3cc44994877e69c3ff83090e from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-83f49f3c3cc44994877e69c3ff83090e
I20260709 12:23:50.862846 31136 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.865271 31002 tablet_copy_source_session.cc:215] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:50.865856 31144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45360
I20260709 12:23:50.866370 31144 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.866688 31144 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:50.882783 32343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83f49f3c3cc44994877e69c3ff83090e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:50.885641 31142 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:23:50.887002 31033 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13959daae3c149d6ae8248d1f3ca6e59. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:50.888918 30983 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:50.889030 32316 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:50.896312 32316 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.891337 30887 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.888983 31144 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:50.908013 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:50.908370 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:50.908957 30991 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:50.916042 30991 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:23:50.918072 32319 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.919092 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:50.908540 31064 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:50.919857 31064 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:50.920607 31064 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:50.908919 31144 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:50.921590 31064 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:50.921025 30983 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.919881 32339 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:50.922858 32339 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:50.923694 32339 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:50.927348 31067 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.930013 32284 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.931267 31067 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:50.932785 32312 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:50.942744 32343 tablet_copy_client.cc:806] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:50.943333 32343 tablet_copy_client.cc:670] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:50.949322 32343 tablet_copy_client.cc:538] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:50.954612 30887 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:50.957450 32343 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:50.969540 30889 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 3 to 4, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.991890 30983 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.996556 32188 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.997000 31064 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:23:51.045135 31136 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:51.081507 32343 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:51.084591 32343 tablet_bootstrap.cc:492] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:51.085386 32343 ts_tablet_manager.cc:1403] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.128s	user 0.020s	sys 0.023s
I20260709 12:23:51.090770 30991 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:51.098271 32359 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:51.134738 32362 ts_tablet_manager.cc:933] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:51.138706 32343 raft_consensus.cc:359] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:51.140082 32343 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:51.140836 32343 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:51.159849 32365 ts_tablet_manager.cc:933] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:51.164409 32362 tablet_copy_client.cc:323] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:51.166574 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 05394269d1a4461bb79d6f93ab25ff4e from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:51.167268 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 05394269d1a4461bb79d6f93ab25ff4e from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-05394269d1a4461bb79d6f93ab25ff4e
I20260709 12:23:51.167445 32365 tablet_copy_client.cc:323] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:51.168730 32343 ts_tablet_manager.cc:1434] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.000s
I20260709 12:23:51.168946 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:51.169500 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-4f57ce098bf541c4b1ca32a9c4b8cec0
I20260709 12:23:51.170341 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-83f49f3c3cc44994877e69c3ff83090e received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:51.170776 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-83f49f3c3cc44994877e69c3ff83090e on tablet 83f49f3c3cc44994877e69c3ff83090e with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:51.178715 31078 tablet_copy_source_session.cc:215] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:51.183465 32362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05394269d1a4461bb79d6f93ab25ff4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:51.184257 31154 tablet_copy_source_session.cc:215] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:51.191382 32365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:51.215430 32365 tablet_copy_client.cc:806] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:51.219184 32365 tablet_copy_client.cc:670] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:51.227963 32365 tablet_copy_client.cc:538] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:51.230644 32362 tablet_copy_client.cc:806] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:51.231241 32362 tablet_copy_client.cc:670] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:51.236881 32362 tablet_copy_client.cc:538] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:51.249213 32362 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:51.253953 31695 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.255460 32316 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.264124 32365 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:51.275403 31144 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.277869 32316 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.285315 32362 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:51.286506 32362 tablet_bootstrap.cc:492] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:51.287365 32362 ts_tablet_manager.cc:1403] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.011s
I20260709 12:23:51.291069 32362 raft_consensus.cc:359] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:51.292009 32362 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:51.292626 32362 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:51.295153 32362 ts_tablet_manager.cc:1434] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:23:51.296816 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-05394269d1a4461bb79d6f93ab25ff4e received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:51.297212 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-05394269d1a4461bb79d6f93ab25ff4e on tablet 05394269d1a4461bb79d6f93ab25ff4e with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:51.373555 32365 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:51.374516 32365 tablet_bootstrap.cc:492] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:51.375195 32365 ts_tablet_manager.cc:1403] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.111s	user 0.015s	sys 0.009s
I20260709 12:23:51.382308 32365 raft_consensus.cc:359] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:51.383162 32365 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:51.383818 32365 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:51.389635 32365 ts_tablet_manager.cc:1434] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:23:51.391474 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-4f57ce098bf541c4b1ca32a9c4b8cec0 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:51.391888 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-4f57ce098bf541c4b1ca32a9c4b8cec0 on tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:51.402918 30991 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:51.405158 32310 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.426203 32365 ts_tablet_manager.cc:933] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:51.426657 31064 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:51.428224 32339 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.436770 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.438616 32365 tablet_copy_client.cc:323] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:51.440732 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 13959daae3c149d6ae8248d1f3ca6e59 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:51.441332 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 13959daae3c149d6ae8248d1f3ca6e59 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-13959daae3c149d6ae8248d1f3ca6e59
I20260709 12:23:51.458606 31078 tablet_copy_source_session.cc:215] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:51.461807 32365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13959daae3c149d6ae8248d1f3ca6e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:51.598821 32365 tablet_copy_client.cc:806] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:51.599398 32365 tablet_copy_client.cc:670] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:51.601089 31695 raft_consensus.cc:1217] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:51.630507 32365 tablet_copy_client.cc:538] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:51.660204 31697 raft_consensus.cc:1217] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:51.670001 31697 raft_consensus.cc:1217] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:51.712865 32365 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:51.746297 31058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76399531471a4790a56d52a6ea052864"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:51.746858 31058 raft_consensus.cc:606] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:51.749415 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:51.749990 32365 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:51.749945 31136 raft_consensus.cc:606] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:51.751515 32365 tablet_bootstrap.cc:492] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:51.752221 32365 ts_tablet_manager.cc:1403] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.040s	user 0.022s	sys 0.011s
I20260709 12:23:51.752548 30983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:51.753145 30983 raft_consensus.cc:606] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:51.754525 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:51.755272 32365 raft_consensus.cc:359] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:51.756090 32365 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:51.756178 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:51.757184 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.756795 32365 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:51.764515 32365 ts_tablet_manager.cc:1434] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:23:51.769665 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-13959daae3c149d6ae8248d1f3ca6e59 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:51.770112 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-13959daae3c149d6ae8248d1f3ca6e59 on tablet 13959daae3c149d6ae8248d1f3ca6e59 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:51.806437 32316 raft_consensus.cc:993] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:51.806887 32316 raft_consensus.cc:1081] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:51.808931 30983 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76399531471a4790a56d52a6ea052864"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:51.809515 30983 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.809916 30983 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:51.826475 30983 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:51.829077 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:51.829667 31058 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:51.829968 31058 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:51.830721 31058 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:51.831789 31058 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:51.832139 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:51.832696 31136 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:51.838577 31058 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:23:51.842464 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:51.846390 32374 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:51.846839 32374 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:51.847628 32374 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:51.854923 31136 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:23:51.855804 30983 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:51.873011 30887 catalog_manager.cc:5697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 3 to 4, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 4 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.993045 31694 raft_consensus.cc:1217] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:52.025714 32316 raft_consensus.cc:1064] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:52.028503 32316 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:52.054718 31694 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:52.056936 30983 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:52.056861 32316 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:52.058540 31136 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:52.060792 32319 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.062933 32319 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.102080 32319 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.104882 31136 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.117677 30983 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.130543 30889 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.149334 31694 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.183198 31064 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:52.222711 31694 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:52.227730 32200 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.229611 31136 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:52.234931 32292 raft_consensus.cc:993] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:52.235462 32292 raft_consensus.cc:1081] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:52.238565 31136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45364
I20260709 12:23:52.239286 31136 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.239647 31136 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.243788 32359 raft_consensus.cc:993] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:52.244246 32359 raft_consensus.cc:1081] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:52.246325 31064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:52.246809 31064 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.247141 31064 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.247895 32200 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.253422 31064 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:52.256211 31064 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:52.263827 32360 raft_consensus.cc:1064] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:52.264988 32264 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.266685 32360 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:52.272632 32292 raft_consensus.cc:1064] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:52.275218 32292 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:52.282716 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 05394269d1a4461bb79d6f93ab25ff4e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:52.292577 30889 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.293905 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:52.294446 31139 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.294742 31139 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:52.295727 31694 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:52.297164 31144 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:52.301452 31064 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:52.295452 31139 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:52.304736 31694 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.305524 31139 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.305197 31144 raft_consensus.cc:2955] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.293284 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:52.307276 30991 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.310485 32374 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.311519 32320 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.314003 30991 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:52.315442 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 05394269d1a4461bb79d6f93ab25ff4e with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:52.316216 32384 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.318728 31058 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:52.304744 31136 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:52.326473 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:52.326954 31058 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.330199 32384 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:52.333033 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:52.333557 30983 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:52.333834 30983 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:52.334583 30983 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:52.335747 30983 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.343056 31058 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:23:52.345556 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:52.346311 32358 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:52.346872 32358 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:52.347711 31136 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:52.347721 32358 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:52.352203 32388 tablet_replica.cc:333] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
W20260709 12:23:52.357129 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 47776.8us
I20260709 12:23:52.357743 32388 raft_consensus.cc:2243] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:52.358501 32388 raft_consensus.cc:2272] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:52.360345 30983 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:23:52.361349 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:52.364259 32316 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.364717 32316 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:52.365491 32316 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:52.387368 31141 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:52.389729 32374 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.369867 32388 ts_tablet_manager.cc:1916] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:52.392385 31139 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:52.406483 31697 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:52.414397 32399 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.432665 30889 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:52.458483 32384 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.457073 30991 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:23:52.457262 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58870.2us
I20260709 12:23:52.469576 31697 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.478559 31067 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:52.496188 30887 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 3, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:52.482821 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48411.3us
I20260709 12:23:52.543733 32264 raft_consensus.cc:1064] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:52.573303 30885 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.484820 32358 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
W20260709 12:23:52.579893 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 83877.4us
I20260709 12:23:52.574415 32388 ts_tablet_manager.cc:1929] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:52.599076 32398 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:23:52.598604 32388 log.cc:1199] T 05394269d1a4461bb79d6f93ab25ff4e P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/05394269d1a4461bb79d6f93ab25ff4e
I20260709 12:23:52.590695 32264 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:52.678062 31064 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.699795 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 05394269d1a4461bb79d6f93ab25ff4e (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
W20260709 12:23:52.720995 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 115303us
I20260709 12:23:52.723847 30983 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:23:52.727608 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 95683.1us
I20260709 12:23:52.708310 32410 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.716698 30992 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.752851 30983 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:52.758572 31139 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.763531 32360 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:23:52.767901 31697 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:52.802367 32320 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.818146 32418 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.825382 31697 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.826748 31064 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:52.829296 30990 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.829993 31694 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.831570 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.844110 32320 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.848408 30889 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.850533 32415 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.853328 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 83f49f3c3cc44994877e69c3ff83090e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:52.858392 31064 raft_consensus.cc:2955] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.864143 31139 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.871335 31695 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:52.872969 32415 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.884490 30990 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:52.888175 31144 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:52.888744 30887 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.891808 32415 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.898200 31067 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:52.908245 31139 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:52.914523 32415 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:52.921092 32416 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:52.923056 31694 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:52.929222 32410 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:52.940956 30991 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:23:52.951839 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 50315.5us
I20260709 12:23:52.946636 32403 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.972479 32420 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:23:52.983179 30983 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.993196 31695 raft_consensus.cc:2955] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:52.998023 32400 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:23:53.003566 32264 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.009413 30991 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.017764 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 83f49f3c3cc44994877e69c3ff83090e with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:53.028343 30887 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 4 to 5, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.044624 31144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:53.045226 31144 raft_consensus.cc:606] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:53.054090 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:53.068943 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:53.071846 30986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:53.072610 30986 raft_consensus.cc:606] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:53.077364 31067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:53.077894 31067 raft_consensus.cc:606] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:53.079763 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:53.082067 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:53.082845 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:53.084192 30889 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.085064 31142 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.089135 31694 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.117197 32433 tablet_replica.cc:333] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:53.122572 32398 raft_consensus.cc:993] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:53.123037 32398 raft_consensus.cc:1081] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:53.108454 32431 tablet_replica.cc:333] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:53.128705 32431 raft_consensus.cc:2243] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:53.129371 32431 raft_consensus.cc:2272] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:53.135748 31067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:53.136301 31067 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:53.136636 31067 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.148329 32431 ts_tablet_manager.cc:1916] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:53.159437 31067 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:53.178479 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:53.179015 30991 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.186308 30991 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:53.188004 31067 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:53.189756 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:53.189936 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:53.190516 31142 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:53.190826 31142 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:53.192840 31142 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:53.193161 32416 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:53.194048 32416 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:53.194921 32416 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:53.194064 31142 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.205503 32433 raft_consensus.cc:2243] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:53.206238 32433 pending_rounds.cc:70] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Trying to abort 1 pending ops.
I20260709 12:23:53.206506 32433 pending_rounds.cc:77] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting op as it isn't in flight: id { term: 2 index: 4 } timestamp: 7305624912957157376 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" old_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } } new_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } } }
I20260709 12:23:53.207774 32433 raft_consensus.cc:2917] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted
I20260709 12:23:53.207881 32431 ts_tablet_manager.cc:1929] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:53.208207 32433 raft_consensus.cc:2889] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:23:53.208695 32433 raft_consensus.cc:2272] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:53.208269 32431 log.cc:1199] T 83f49f3c3cc44994877e69c3ff83090e P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/83f49f3c3cc44994877e69c3ff83090e
I20260709 12:23:53.211817 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 83f49f3c3cc44994877e69c3ff83090e (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:53.212940 32433 ts_tablet_manager.cc:1916] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:53.223731 31067 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:53.229425 32433 ts_tablet_manager.cc:1929] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:53.230798 32433 log.cc:1199] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/4f57ce098bf541c4b1ca32a9c4b8cec0
I20260709 12:23:53.231034 30887 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:53.232977 31694 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.223758 31142 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:53.234478 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet 4f57ce098bf541c4b1ca32a9c4b8cec0 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:53.236404 32416 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:53.237715 30991 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.239965 32416 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:53.246668 32413 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.251137 30991 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.260244 31694 raft_consensus.cc:2955] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.262809 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 13959daae3c149d6ae8248d1f3ca6e59 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:53.270141 30887 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 4 to 5, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:53.287125 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 13959daae3c149d6ae8248d1f3ca6e59 with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:53.288030 32431 tablet_replica.cc:333] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:53.288934 32431 raft_consensus.cc:2243] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:53.289665 32431 raft_consensus.cc:2272] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:53.297001 32431 ts_tablet_manager.cc:1916] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:53.316311 32431 ts_tablet_manager.cc:1929] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:53.316798 32431 log.cc:1199] T 13959daae3c149d6ae8248d1f3ca6e59 P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/13959daae3c149d6ae8248d1f3ca6e59
I20260709 12:23:53.319464 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 13959daae3c149d6ae8248d1f3ca6e59 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:53.384171 32418 raft_consensus.cc:993] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:53.385164 32414 raft_consensus.cc:1081] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:53.388880 31694 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51362
I20260709 12:23:53.389381 31694 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:53.389706 31694 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.395041 32416 raft_consensus.cc:993] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:53.395546 32416 raft_consensus.cc:1081] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:53.395815 31694 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:53.397073 30991 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:53.397676 30991 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:53.398049 30991 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:53.399116 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:53.399655 30986 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:53.399926 30986 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:53.400624 30986 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:53.401659 30986 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.401938 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:53.402469 31067 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:53.403784 30991 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:53.405557 30991 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:53.406622 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:53.407158 31060 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:53.407189 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:53.407567 31060 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:53.407815 31142 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:53.408344 31060 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:53.409449 31060 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:53.412237 30986 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:53.413578 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:53.414323 32188 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:53.414783 32188 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:53.416026 32188 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:53.416191 31067 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:23:53.417338 31060 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:23:53.417696 31694 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:53.418952 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:53.417559 31142 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:23:53.419698 32418 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:53.420370 32418 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:53.421553 32418 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:53.427553 30889 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:53.432401 30885 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 2 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:53.610728 30986 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:53.611904 32413 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:53.620030 31136 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:53.621193 32416 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:53.843534 31060 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:53.844668 32414 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:53.852324 31136 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:53.854874 32389 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:53.929013 30991 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.930303 32402 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:53.949239 31060 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:53.950846 32188 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:54.201644 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.239915 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:54.240513 31060 raft_consensus.cc:606] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:54.242789 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "205fb3f3a3844073a49da13542da2989"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:54.243283 31136 raft_consensus.cc:606] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:54.245375 30986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:54.245811 30986 raft_consensus.cc:606] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:54.246866 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:54.248023 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:54.248698 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.250501 32420 raft_consensus.cc:993] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:54.250947 32360 raft_consensus.cc:1081] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:54.252172 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "205fb3f3a3844073a49da13542da2989"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:54.252650 31060 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:54.252970 31060 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.261073 31060 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:54.266739 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:54.267242 30986 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.274240 30986 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:54.275316 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:54.276281 32416 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:54.277562 31060 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:54.278111 32416 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:54.279592 32416 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:54.282104 32436 raft_consensus.cc:993] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:54.282208 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:54.282696 32436 raft_consensus.cc:1081] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:54.282848 31136 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:54.283200 31136 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:54.283908 31136 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:54.285336 31142 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45364
I20260709 12:23:54.285854 31142 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:54.286177 31142 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.293915 31142 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:54.294550 31136 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.296483 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:54.300212 30986 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:54.300590 30986 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:54.301074 31136 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:54.301365 30986 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:54.302466 30986 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.303987 30889 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:54.307662 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:23:54.308192 31694 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:54.325634 31694 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:54.326993 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:54.328030 32360 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:54.328564 32360 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:54.328684 31142 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:23:54.329432 32360 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:54.330833 30986 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:54.347641 30889 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:54.384557 31060 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:54.390477 31142 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:54.392019 30991 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:54.392177 32416 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:23:54.392043 31033 consensus_peers.cc:597] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 48047b2b139146c38a821328cf7bd148. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:54.396090 32416 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:54.403627 32413 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.405256 31142 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.407528 30986 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.422607 31142 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:54.422503 30889 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 3, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:23:54.437868 31109 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a2c77b15766d44f585ab3cb57444a324. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:54.438575 30986 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:54.440044 32420 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:54.440491 31060 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:54.443092 32420 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:54.449822 32360 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.459373 30991 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.466678 31060 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.468541 31064 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:54.481751 30889 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:54.491647 31144 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:54.495139 32416 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:54.497038 31033 consensus_peers.cc:597] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d63b31721b1434ea576ac0fab297089. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:54.502730 30991 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:54.504040 32416 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:54.507282 32413 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.509111 31142 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.535476 30986 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:54.545503 30887 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:23:54.651959 30958 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41116db50e8c4b9598abe43f5d36bf12. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:54.652136 31142 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:54.653750 32436 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:54.662446 31067 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:54.664593 32468 raft_consensus.cc:993] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:54.665059 32468 raft_consensus.cc:1081] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:54.666775 30986 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:54.667291 30986 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:54.667649 30986 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.673539 30984 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.681668 32389 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:54.708340 32436 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.717443 30986 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:54.717975 31067 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.718575 31142 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:54.719422 30986 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:23:54.720140 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:54.726926 31064 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:54.727250 31064 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:54.727221 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:23:54.727907 31694 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.741566 31694 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:23:54.742729 31064 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:54.743178 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:54.742553 30887 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:54.743918 31064 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:54.745445 32473 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:54.748057 30990 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:54.749341 32410 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:54.751881 32473 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:54.752774 32473 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:54.765059 31064 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:23:54.780656 31694 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:54.782785 32410 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:54.800568 30885 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 2 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:54.822263 30986 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:54.823697 32468 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:54.835359 31142 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:54.836884 32468 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:54.877173 32483 ts_tablet_manager.cc:933] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:54.890362 32483 tablet_copy_client.cc:323] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:54.892905 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 48047b2b139146c38a821328cf7bd148 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:54.893481 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 48047b2b139146c38a821328cf7bd148 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-48047b2b139146c38a821328cf7bd148
I20260709 12:23:54.902964 32486 ts_tablet_manager.cc:933] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:54.908016 31078 tablet_copy_source_session.cc:215] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:54.912207 32486 tablet_copy_client.cc:323] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:54.912815 32487 ts_tablet_manager.cc:933] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:54.913751 32483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48047b2b139146c38a821328cf7bd148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:54.913801 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 0d63b31721b1434ea576ac0fab297089 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:54.914552 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 0d63b31721b1434ea576ac0fab297089 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-0d63b31721b1434ea576ac0fab297089
I20260709 12:23:54.915555 32487 tablet_copy_client.cc:323] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:54.918833 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet a2c77b15766d44f585ab3cb57444a324 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:54.919346 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet a2c77b15766d44f585ab3cb57444a324 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-a2c77b15766d44f585ab3cb57444a324
I20260709 12:23:54.930671 31154 tablet_copy_source_session.cc:215] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:54.933544 32483 tablet_copy_client.cc:806] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:54.934293 32483 tablet_copy_client.cc:670] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:54.935945 32487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2c77b15766d44f585ab3cb57444a324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:54.969023 31078 tablet_copy_source_session.cc:215] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:55.000406 32486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d63b31721b1434ea576ac0fab297089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:55.021088 32483 tablet_copy_client.cc:538] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:55.066520 32483 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:55.077682 32486 tablet_copy_client.cc:806] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:55.089473 32486 tablet_copy_client.cc:670] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:55.093573 32490 ts_tablet_manager.cc:933] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:55.088639 32487 tablet_copy_client.cc:806] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:55.101990 32486 tablet_copy_client.cc:538] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:55.110544 32487 tablet_copy_client.cc:670] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:55.121660 32487 tablet_copy_client.cc:538] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:55.134388 32486 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:55.144788 32487 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:55.147110 32490 tablet_copy_client.cc:323] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:55.149500 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet 41116db50e8c4b9598abe43f5d36bf12 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:55.150344 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet 41116db50e8c4b9598abe43f5d36bf12 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-41116db50e8c4b9598abe43f5d36bf12
I20260709 12:23:55.182128 31002 tablet_copy_source_session.cc:215] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:55.231639 31067 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:55.233741 32389 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:55.247784 32483 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:55.248792 32483 tablet_bootstrap.cc:492] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:55.249713 32483 ts_tablet_manager.cc:1403] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.184s	user 0.021s	sys 0.024s
I20260709 12:23:55.253693 32483 raft_consensus.cc:359] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:55.254720 32483 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:55.257570 32483 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:55.266250 32483 ts_tablet_manager.cc:1434] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:23:55.275602 32490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41116db50e8c4b9598abe43f5d36bf12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:55.282480 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-48047b2b139146c38a821328cf7bd148 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:55.283037 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-48047b2b139146c38a821328cf7bd148 on tablet 48047b2b139146c38a821328cf7bd148 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:55.290021 31695 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:55.315140 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:55.330754 32496 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:23:55.347724 32487 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:55.360379 32487 tablet_bootstrap.cc:492] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:55.361245 32487 ts_tablet_manager.cc:1403] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.217s	user 0.020s	sys 0.017s
I20260709 12:23:55.374063 32486 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:55.357466 32490 tablet_copy_client.cc:806] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:55.378811 32490 tablet_copy_client.cc:670] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:55.385083 32490 tablet_copy_client.cc:538] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:55.374948 32486 tablet_bootstrap.cc:492] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:55.387564 32486 ts_tablet_manager.cc:1403] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.273s	user 0.053s	sys 0.053s
I20260709 12:23:55.390202 32486 raft_consensus.cc:359] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:55.390988 32486 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:55.391574 32486 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:55.391629 32487 raft_consensus.cc:359] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:55.392550 32487 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:55.393587 32486 ts_tablet_manager.cc:1434] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:55.393226 32487 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:55.394470 32490 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:55.395218 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-0d63b31721b1434ea576ac0fab297089 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:55.395632 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-0d63b31721b1434ea576ac0fab297089 on tablet 0d63b31721b1434ea576ac0fab297089 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:55.400629 32487 ts_tablet_manager.cc:1434] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.001s
I20260709 12:23:55.403326 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-a2c77b15766d44f585ab3cb57444a324 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:55.403810 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-a2c77b15766d44f585ab3cb57444a324 on tablet a2c77b15766d44f585ab3cb57444a324 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:55.426095 31695 raft_consensus.cc:1217] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:55.433949 32490 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:55.441094 32490 tablet_bootstrap.cc:492] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:55.441776 32490 ts_tablet_manager.cc:1403] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.014s
I20260709 12:23:55.445505 32490 raft_consensus.cc:359] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:55.446354 32490 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:55.447026 32490 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:55.450017 32490 ts_tablet_manager.cc:1434] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.007s
I20260709 12:23:55.451784 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-41116db50e8c4b9598abe43f5d36bf12 received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:55.452239 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-41116db50e8c4b9598abe43f5d36bf12 on tablet 41116db50e8c4b9598abe43f5d36bf12 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:55.466876 31144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:55.467510 31144 raft_consensus.cc:606] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:55.472222 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:55.472729 31060 raft_consensus.cc:606] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:55.475451 30986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:55.475886 30986 raft_consensus.cc:606] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:55.476969 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:55.478032 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:55.478837 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:55.535424 31695 raft_consensus.cc:1217] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:55.609483 32498 raft_consensus.cc:993] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:55.610080 32360 raft_consensus.cc:1081] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:55.611544 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:55.612040 31060 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.612370 31060 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.619457 31060 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:55.621537 31060 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:23:55.623159 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:23:55.623914 31695 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.630656 31695 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:55.631412 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:55.631999 31144 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:55.632306 31144 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:55.632236 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:23:55.633337 31144 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:55.634651 31144 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.636258 32264 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:55.636701 32264 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:55.637459 32264 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:55.641223 31144 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:55.664639 30889 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:55.748003 31695 raft_consensus.cc:1217] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:55.812042 32468 raft_consensus.cc:993] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:55.812521 32264 raft_consensus.cc:1081] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:55.813961 30986 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:55.814432 30986 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:55.814790 30986 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.820801 30986 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:55.823053 30986 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:55.823141 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:55.823520 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:55.823845 31060 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:55.824131 31142 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.824185 31060 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:55.824993 31060 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:55.825811 31060 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:55.832036 31060 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:55.833277 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:55.834172 32414 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:55.834700 31142 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:55.842446 31697 raft_consensus.cc:1217] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:55.835376 32414 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:55.847885 32414 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:55.861543 30886 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:55.990178 32414 raft_consensus.cc:1064] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:55.990401 32360 raft_consensus.cc:1064] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:55.992766 32360 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:55.992970 32414 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.002466 31695 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.004701 31064 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.006778 31067 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.008991 32498 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.010249 30986 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.011644 32498 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.016127 32509 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:56.018631 31142 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.022907 32389 raft_consensus.cc:993] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:56.027732 31695 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.031031 32512 raft_consensus.cc:1081] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:56.032750 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:23:56.024780 32496 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:23:56.033514 31060 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:56.033951 31060 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.039822 32498 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.051508 31060 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.053364 31060 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:23:56.053537 31136 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:56.056358 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:56.056881 30986 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:56.057188 30986 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:56.057364 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:23:56.057922 31697 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.057973 30986 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.060055 30986 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.064148 32416 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.072235 31697 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:56.074823 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:23:56.075613 32416 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:56.076117 32416 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:56.076560 32414 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.080200 30986 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:56.077023 32416 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.101491 32498 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.111725 30990 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.124655 31695 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.127852 30886 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:56.137604 31058 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.143952 32509 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.165663 31144 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.169706 31694 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:56.170948 31697 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.172410 32468 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.189726 30886 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.232815 31067 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.241809 31058 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:56.243844 32414 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:56.246930 30885 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.260810 31144 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.270735 31694 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:56.274832 32360 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:56.293625 32498 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.283849 30983 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.298769 31694 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.302820 30990 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:56.324777 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a2c77b15766d44f585ab3cb57444a324 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:56.333902 32508 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:56.353077 32416 raft_consensus.cc:1064] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:56.365036 32416 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.368444 30889 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.376648 32468 raft_consensus.cc:1064] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:56.379123 32468 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.384373 31695 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:23:56.387918 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 55388.8us
I20260709 12:23:56.399426 31144 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.403764 31697 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:56.404672 32416 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.407294 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet a2c77b15766d44f585ab3cb57444a324 with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:56.411319 32530 tablet_replica.cc:333] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:56.413130 32530 raft_consensus.cc:2243] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.415055 31144 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:56.416582 31695 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.418104 32468 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:56.420584 32468 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:56.422850 32416 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:56.426530 32530 raft_consensus.cc:2272] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.434286 32530 ts_tablet_manager.cc:1916] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:56.446508 31136 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:56.449791 30983 raft_consensus.cc:2955] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.452739 32468 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.463150 31694 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.469049 31142 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.477176 30886 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 3 to 4, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.491276 32530 ts_tablet_manager.cc:1929] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:56.491894 32530 log.cc:1199] T a2c77b15766d44f585ab3cb57444a324 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/a2c77b15766d44f585ab3cb57444a324
W20260709 12:23:56.504456 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 54955.5us
I20260709 12:23:56.508944 31060 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.513285 31142 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:56.517976 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet a2c77b15766d44f585ab3cb57444a324 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:56.520417 30986 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:56.505247 32496 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:23:56.523133 32509 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:56.520519 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.524101 30986 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:56.538702 32509 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:56.545177 31142 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:56.544763 32416 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:56.547302 31698 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:56.557606 32528 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.560465 32512 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.567580 31136 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.568671 32416 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.579088 31698 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.579820 31135 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.587723 30887 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.591053 32468 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.591435 31694 raft_consensus.cc:2955] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.600878 31697 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:56.620311 32416 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.629272 32468 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.630211 30889 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.651192 31139 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.663447 31698 raft_consensus.cc:2955] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.652802 31060 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:56.682170 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 48047b2b139146c38a821328cf7bd148 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:56.694818 31697 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:56.701545 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 41116db50e8c4b9598abe43f5d36bf12 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:56.705204 31139 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:56.710012 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet 41116db50e8c4b9598abe43f5d36bf12 with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:23:56.711009 32530 tablet_replica.cc:333] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:23:56.712134 32530 raft_consensus.cc:2243] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.711906 32468 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:56.712790 32530 raft_consensus.cc:2272] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.716509 32530 ts_tablet_manager.cc:1916] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:56.717496 30887 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 4 to 5, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.719635 32468 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:56.724536 32468 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.726972 31697 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.734928 30984 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:56.743124 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 48047b2b139146c38a821328cf7bd148 with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:56.744761 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0d63b31721b1434ea576ac0fab297089 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:56.749577 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:56.750156 31060 raft_consensus.cc:606] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:56.751008 32538 tablet_replica.cc:333] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:56.752187 32538 raft_consensus.cc:2243] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.753401 32530 ts_tablet_manager.cc:1929] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:56.754439 32468 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:56.757999 32538 raft_consensus.cc:2272] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.757905 32530 log.cc:1199] T 41116db50e8c4b9598abe43f5d36bf12 P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/41116db50e8c4b9598abe43f5d36bf12
I20260709 12:23:56.766050 30984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:56.766620 30984 raft_consensus.cc:606] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:56.770041 32538 ts_tablet_manager.cc:1916] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:56.771118 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:56.771673 31060 raft_consensus.cc:606] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:23:56.769059 31136 raft_consensus.cc:2955] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:56.777359 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:56.765326 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet 41116db50e8c4b9598abe43f5d36bf12 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:56.777885 31060 raft_consensus.cc:606] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:56.779426 30885 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 3 to 4, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 1 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:56.784785 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:56.788331 32538 ts_tablet_manager.cc:1929] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:23:56.788818 32538 log.cc:1199] T 48047b2b139146c38a821328cf7bd148 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/48047b2b139146c38a821328cf7bd148
I20260709 12:23:56.790472 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 48047b2b139146c38a821328cf7bd148 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:56.787525 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:56.793215 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.815918 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d63b31721b1434ea576ac0fab297089 with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:56.816892 32538 tablet_replica.cc:333] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:56.817934 32538 raft_consensus.cc:2243] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.818548 32538 pending_rounds.cc:70] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: Trying to abort 1 pending ops.
I20260709 12:23:56.818807 32538 pending_rounds.cc:77] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305624929743777792 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "0d63b31721b1434ea576ac0fab297089" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } } }
I20260709 12:23:56.819813 32538 raft_consensus.cc:2917] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 12:23:56.820171 32538 raft_consensus.cc:2889] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:23:56.820410 32538 raft_consensus.cc:2272] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.824610 32538 ts_tablet_manager.cc:1916] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:56.837026 32538 ts_tablet_manager.cc:1929] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:56.837311 32538 log.cc:1199] T 0d63b31721b1434ea576ac0fab297089 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/0d63b31721b1434ea576ac0fab297089
I20260709 12:23:56.838752 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet 0d63b31721b1434ea576ac0fab297089 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:56.856608 32509 raft_consensus.cc:993] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:56.857108 32389 raft_consensus.cc:1081] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:56.858428 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:23:56.858836 31060 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:56.859102 31060 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.864223 31060 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:56.865782 31060 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:56.866693 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:56.866705 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:56.867192 30990 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:56.867316 31136 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.867509 30990 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:56.868153 30990 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:56.868968 30990 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.872911 31136 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:56.875586 30990 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:56.875771 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:56.876564 32264 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:56.876983 32264 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:56.877673 32264 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:56.886603 30889 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:56.903177 32264 raft_consensus.cc:993] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:56.903587 32528 raft_consensus.cc:1081] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:56.905059 30990 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:56.905460 30990 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:56.905694 30990 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.910178 30990 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:56.911449 30990 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:56.912190 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:56.912340 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:56.912675 31060 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:56.912853 31136 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.913040 31060 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:56.913849 31060 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:56.916791 31060 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.918874 31136 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:56.919760 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:56.920473 32389 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:56.920920 32389 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:56.921720 32389 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:56.923949 31060 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:56.933087 30889 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:57.119900 32264 raft_consensus.cc:993] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:57.120529 32264 raft_consensus.cc:1081] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:57.122434 31139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45360
I20260709 12:23:57.122920 31139 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.123237 31139 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.129691 31139 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:57.131393 31139 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:23:57.132136 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:57.132725 31067 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:57.132974 31067 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:57.133567 31067 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:57.134474 31067 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.136272 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:23:57.136790 31698 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.141913 31067 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:23:57.142009 31698 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:23:57.143136 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:57.144197 32526 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:57.145287 32526 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:57.146211 32526 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:57.147938 32528 raft_consensus.cc:993] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:57.148401 32264 raft_consensus.cc:1081] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:57.149756 31139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45360
I20260709 12:23:57.150182 31139 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:57.150566 31139 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.155972 31139 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:57.157497 31139 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:57.157191 30889 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:57.158483 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:57.159049 30990 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.163178 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:57.163666 31060 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:57.163952 31060 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:57.164661 31060 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:57.165843 31060 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.166535 30990 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:57.171353 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:57.172662 31060 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:23:57.172019 32526 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:57.173887 32526 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:57.174850 32526 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:57.185591 30889 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:57.339620 31060 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:57.340684 32509 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:57.354184 31136 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:57.356055 32496 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:57.432242 31139 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:57.433429 32264 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:57.441996 30990 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:57.443261 32264 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:57.632714 31694 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:57.633910 32526 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:57.637300 31067 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:57.638834 32526 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:57.658900 30990 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:57.659279 31060 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:57.661427 32526 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:57.668815 32554 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:57.859478 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:57.873111 30990 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:57.881788 31139 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:57.882678 30958 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2439e7934b98446ca4e7d82381e8a499. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:57.883188 31060 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:57.884598 32496 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:57.887703 32496 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:57.894580 32560 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:57.901598 31060 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:57.915877 30889 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:57.923920 31058 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:57.925943 30990 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:57.926496 30990 raft_consensus.cc:606] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:57.929244 31144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:57.929790 31144 raft_consensus.cc:606] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:57.928470 31139 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:57.944058 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:57.944847 31060 raft_consensus.cc:606] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:23:57.952025 31033 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 205fb3f3a3844073a49da13542da2989. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:57.952489 30990 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:57.952924 31139 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:57.953945 32264 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:57.955363 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:57.955916 31136 raft_consensus.cc:606] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:57.956884 32264 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:57.957252 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:57.959022 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:57.959728 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:57.979434 32468 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:57.981041 30984 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:57.983677 31139 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.001220 30888 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 4, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.007937 31139 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:58.022324 30990 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:23:58.023085 31058 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:23:58.023846 32360 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:58.024583 32526 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:58.026332 31109 consensus_peers.cc:597] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e39858333670419fb20dc8adde042dd2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:58.043027 32360 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.056227 30984 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.065953 31060 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.069550 31139 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:58.073300 30888 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 5, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.080777 30984 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:58.081370 31109 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d50d1465e4b84136bf90de700a93ef2c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:58.082249 32360 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:58.083702 31060 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:58.086442 32360 raft_consensus.cc:993] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:58.086936 32554 raft_consensus.cc:1081] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:58.087246 32525 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:58.089550 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:58.090314 31060 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:58.090750 31060 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:58.096004 32360 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.097505 31060 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.097704 31067 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.099922 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:58.100464 30984 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:58.100597 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:58.101135 31139 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:58.101423 31139 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:58.102126 31139 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.101832 30990 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.103181 31139 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:58.107120 30984 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:58.108175 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:58.108959 32264 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:58.109355 32264 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:58.109777 31139 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:23:58.110206 32264 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.110739 31060 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:58.125577 30885 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: config changed from index -1 to 2, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:23:58.135143 30888 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.306708 32496 raft_consensus.cc:993] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:58.307242 32559 raft_consensus.cc:1081] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:58.309764 31139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45364
I20260709 12:23:58.310382 31139 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:58.310768 31139 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.316758 31139 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.319918 31139 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:58.321698 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:58.322070 32554 raft_consensus.cc:993] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:58.322317 31060 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.322579 32554 raft_consensus.cc:1081] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:23:58.324203 31698 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51360
I20260709 12:23:58.324664 31698 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:58.324970 31698 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.332394 31698 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:58.322557 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:58.336100 30990 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:58.336673 30990 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:58.338044 31698 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:58.328522 31060 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:23:58.338944 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:58.339474 30984 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.337469 30990 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.339833 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:23:58.340372 30990 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.340942 32526 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:58.343568 32526 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:58.344401 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:58.345140 31139 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:58.345513 31139 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:58.345360 32526 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.346498 31139 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:58.349489 31139 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.351346 30984 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:23:58.353626 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:58.354390 32564 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:58.354871 32564 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:23:58.355669 32564 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:58.356221 30990 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:23:58.374418 31139 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:23:58.384430 30889 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 3, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 3 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:58.389621 32468 raft_consensus.cc:993] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:58.390096 32468 raft_consensus.cc:1081] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:23:58.391708 30990 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49530
I20260709 12:23:58.391285 30888 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 3, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:58.392390 30990 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:58.392781 30990 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:58.399641 30990 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.401696 30990 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:58.402304 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:58.402843 31060 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:58.403007 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:58.403162 31060 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:58.403512 31139 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:58.403873 31060 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.404981 31060 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:58.409711 31139 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:58.410971 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:58.411732 32559 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:58.412170 32559 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:58.412972 32559 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:58.413666 31060 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:23:58.425908 32580 ts_tablet_manager.cc:933] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:58.430819 32580 tablet_copy_client.cc:323] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:23:58.432178 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet 205fb3f3a3844073a49da13542da2989 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:23:58.432605 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet 205fb3f3a3844073a49da13542da2989 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-205fb3f3a3844073a49da13542da2989
I20260709 12:23:58.432989 30888 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.438772 31078 tablet_copy_source_session.cc:215] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:58.442160 32580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 205fb3f3a3844073a49da13542da2989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:58.446177 32582 ts_tablet_manager.cc:933] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:58.456496 32582 tablet_copy_client.cc:323] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:58.458671 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet e39858333670419fb20dc8adde042dd2 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:58.459175 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet e39858333670419fb20dc8adde042dd2 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-e39858333670419fb20dc8adde042dd2
I20260709 12:23:58.461084 32580 tablet_copy_client.cc:806] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:58.461620 32580 tablet_copy_client.cc:670] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:58.465332 32580 tablet_copy_client.cc:538] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:58.465613 31154 tablet_copy_source_session.cc:215] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:58.469137 32582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e39858333670419fb20dc8adde042dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:58.474416 32580 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:58.484731 32582 tablet_copy_client.cc:806] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:58.485359 32582 tablet_copy_client.cc:670] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:58.489393 32582 tablet_copy_client.cc:538] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:58.501821 32582 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:58.521113 32585 ts_tablet_manager.cc:933] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:23:58.533504 32580 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:58.534665 32580 tablet_bootstrap.cc:492] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:58.535547 32580 ts_tablet_manager.cc:1403] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.061s	user 0.026s	sys 0.012s
I20260709 12:23:58.547942 32585 tablet_copy_client.cc:323] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.65:34803
I20260709 12:23:58.549491 32580 raft_consensus.cc:359] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:58.550364 32580 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:58.552879 31002 tablet_copy_service.cc:140] P e9cccf7eb2a545ef8c2169b67ff261bf: Received BeginTabletCopySession request for tablet 2439e7934b98446ca4e7d82381e8a499 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:41814)
I20260709 12:23:58.553433 31002 tablet_copy_service.cc:161] P e9cccf7eb2a545ef8c2169b67ff261bf: Beginning new tablet copy session on tablet 2439e7934b98446ca4e7d82381e8a499 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:41814: session id = a517aadb0ae04b89a13779d7898dfa0c-2439e7934b98446ca4e7d82381e8a499
I20260709 12:23:58.551899 32580 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:58.575814 32580 ts_tablet_manager.cc:1434] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.003s
I20260709 12:23:58.579128 31002 tablet_copy_source_session.cc:215] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:58.581926 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-205fb3f3a3844073a49da13542da2989 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:23:58.589349 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-205fb3f3a3844073a49da13542da2989 on tablet 205fb3f3a3844073a49da13542da2989 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:58.587203 32588 ts_tablet_manager.cc:933] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:58.599041 32585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2439e7934b98446ca4e7d82381e8a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:58.607384 32582 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:58.608853 32582 tablet_bootstrap.cc:492] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:58.609525 32582 ts_tablet_manager.cc:1403] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.108s	user 0.034s	sys 0.017s
I20260709 12:23:58.612644 32582 raft_consensus.cc:359] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:58.613416 32582 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:58.614055 32582 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:58.620859 32582 ts_tablet_manager.cc:1434] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 12:23:58.622750 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-e39858333670419fb20dc8adde042dd2 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:58.623137 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-e39858333670419fb20dc8adde042dd2 on tablet e39858333670419fb20dc8adde042dd2 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:58.641439 30990 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:58.642635 32264 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:58.663992 31139 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:58.670751 32528 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:58.766595 31144 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:58.768831 32465 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:23:58.775157 30984 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:58.776476 32564 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:58.787771 30986 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.789366 32526 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:58.823812 31067 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.825593 32526 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:58.859460 32588 tablet_copy_client.cc:323] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:23:58.861634 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet d50d1465e4b84136bf90de700a93ef2c from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:23:58.862124 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet d50d1465e4b84136bf90de700a93ef2c from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-d50d1465e4b84136bf90de700a93ef2c
I20260709 12:23:58.871989 31154 tablet_copy_source_session.cc:215] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:58.873888 31694 raft_consensus.cc:1217] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEARNER]: Deduplicated request from leader. Original: 4.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 12:23:58.887130 32588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d50d1465e4b84136bf90de700a93ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:58.912379 32588 tablet_copy_client.cc:806] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:58.912995 32588 tablet_copy_client.cc:670] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:58.916671 31058 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:58.917976 32509 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:58.918759 32588 tablet_copy_client.cc:538] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:58.943367 31135 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:58.956142 32600 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:23:58.966301 32595 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
W20260709 12:23:58.980495 32603 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.056s	user 0.002s	sys 0.000s
I20260709 12:23:58.982983 32585 tablet_copy_client.cc:806] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:58.983840 32585 tablet_copy_client.cc:670] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:59.007113 32588 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:59.023195 32585 tablet_copy_client.cc:538] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:59.038937 31698 raft_consensus.cc:1217] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:23:59.059042 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:59.063838 32585 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:23:59.093812 32588 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:59.096791 32588 tablet_bootstrap.cc:492] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:59.097517 32588 ts_tablet_manager.cc:1403] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.091s	user 0.015s	sys 0.007s
I20260709 12:23:59.123972 32585 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:59.123653 32588 raft_consensus.cc:359] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:59.124835 32588 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:59.125074 32585 tablet_bootstrap.cc:492] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:23:59.125886 32585 ts_tablet_manager.cc:1403] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.062s	user 0.025s	sys 0.004s
I20260709 12:23:59.125560 32588 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:59.128204 32588 ts_tablet_manager.cc:1434] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.002s
I20260709 12:23:59.129256 32585 raft_consensus.cc:359] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:59.130333 32585 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:23:59.130990 32585 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:23:59.132614 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-d50d1465e4b84136bf90de700a93ef2c received from {username='slave'} at 127.0.0.1:42006
I20260709 12:23:59.133031 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-d50d1465e4b84136bf90de700a93ef2c on tablet d50d1465e4b84136bf90de700a93ef2c with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:59.135141 32585 ts_tablet_manager.cc:1434] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:23:59.137010 31002 tablet_copy_service.cc:342] P e9cccf7eb2a545ef8c2169b67ff261bf: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-2439e7934b98446ca4e7d82381e8a499 received from {username='slave'} at 127.0.0.1:41814
I20260709 12:23:59.137401 31002 tablet_copy_service.cc:434] P e9cccf7eb2a545ef8c2169b67ff261bf: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-2439e7934b98446ca4e7d82381e8a499 on tablet 2439e7934b98446ca4e7d82381e8a499 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:23:59.178642 31144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:23:59.179226 31144 raft_consensus.cc:606] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:23:59.183807 31067 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:23:59.184307 31067 raft_consensus.cc:606] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:59.186735 30986 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60048
I20260709 12:23:59.187240 30986 raft_consensus.cc:606] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:23:59.188347 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:59.189918 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:59.190613 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:59.389770 32606 raft_consensus.cc:1064] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:59.389760 32264 raft_consensus.cc:993] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:59.390540 32528 raft_consensus.cc:1081] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:59.392057 31144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45360
I20260709 12:23:59.392562 31144 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.392925 31144 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.392445 32606 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:59.399976 31144 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.401796 31067 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:23:59.403119 32360 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:59.403381 31144 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:59.404554 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:59.405093 31067 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:59.405364 31067 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:59.406019 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:59.406554 30986 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.406719 31694 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:23:59.406029 31067 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.410404 31067 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.410755 32554 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:59.413652 30986 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:23:59.414916 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:59.417732 32606 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:59.418252 32606 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:59.419086 32606 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.423691 30986 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:23:59.425711 32554 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:59.430289 32360 raft_consensus.cc:993] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:59.432478 32360 raft_consensus.cc:1081] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:23:59.429463 31067 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:23:59.442629 31067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:23:59.443166 31067 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.443603 31067 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.452981 31067 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.456733 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:59.457239 30984 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.461345 32360 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.464807 31058 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.468405 30984 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:59.470808 31067 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:23:59.472440 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:59.473233 32528 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:59.473670 32528 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:23:59.474462 32528 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.475307 31694 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.475327 30984 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.478497 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:23:59.479035 31144 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:59.479323 31144 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:59.480053 31144 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.481184 31144 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.493885 30888 catalog_manager.cc:5697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:59.495894 31144 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:23:59.497022 31698 raft_consensus.cc:1217] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:59.505066 30889 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 5 to 6, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.512399 30885 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.563153 31694 raft_consensus.cc:1217] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:59.567584 31141 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:59.572407 31694 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:23:59.573787 32360 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:59.572824 31058 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:23:59.581503 32606 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:59.588069 32606 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.594400 31694 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.596782 31058 raft_consensus.cc:2955] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.603833 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e39858333670419fb20dc8adde042dd2 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:59.610778 30887 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 6 to 7, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:59.622498 30980 tablet_service.cc:1558] Processing DeleteTablet for tablet e39858333670419fb20dc8adde042dd2 with delete_type TABLET_DATA_TOMBSTONED (TS e9cccf7eb2a545ef8c2169b67ff261bf not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:49514
I20260709 12:23:59.628749 32496 raft_consensus.cc:993] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:59.629725 32496 raft_consensus.cc:1081] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:23:59.640254 31141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45364
I20260709 12:23:59.640748 31141 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.641059 31141 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.647933 31141 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.656288 32625 tablet_replica.cc:333] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:23:59.663861 32528 raft_consensus.cc:1064] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:59.666795 31141 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:23:59.666873 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:23:59.667533 30986 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:59.667804 30986 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:23:59.668438 30986 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.666527 32528 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:59.670071 30986 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.672462 32625 raft_consensus.cc:2243] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:59.673269 32625 raft_consensus.cc:2272] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:59.676950 32625 ts_tablet_manager.cc:1916] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:59.694959 32625 ts_tablet_manager.cc:1929] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.6
I20260709 12:23:59.695417 32625 log.cc:1199] T e39858333670419fb20dc8adde042dd2 P e9cccf7eb2a545ef8c2169b67ff261bf: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/wals/e39858333670419fb20dc8adde042dd2
I20260709 12:23:59.697041 30876 catalog_manager.cc:5028] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): tablet e39858333670419fb20dc8adde042dd2 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:59.700295 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:23:59.700932 31058 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.702991 30986 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:23:59.704077 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:23:59.704897 32554 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:59.705332 32554 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:23:59.706097 32554 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:23:59.707674 31058 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:23:59.712291 31694 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:59.714516 32468 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:59.713995 31141 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:59.719112 32468 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:59.726594 30984 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:59.730698 32264 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:59.732012 30885 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 3 to 4, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:59.744208 32630 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.745522 31694 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.754072 30986 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.760207 30885 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 4 to 5, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:59.765362 31141 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.774742 31058 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:23:59.775821 32630 consensus_queue.cc:1175] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 5 } }
I20260709 12:23:59.789265 30986 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:59.790769 32528 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:59.792695 31694 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:59.794099 32528 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:59.802135 32630 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.803736 31694 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.805599 30986 raft_consensus.cc:2955] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:23:59.812711 30873 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 205fb3f3a3844073a49da13542da2989 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:59.828635 30885 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index 5 to 6, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:59.839763 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 205fb3f3a3844073a49da13542da2989 with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45358
I20260709 12:23:59.853881 32632 tablet_replica.cc:333] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:23:59.855047 32632 raft_consensus.cc:2243] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:59.855873 32632 raft_consensus.cc:2272] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:59.859994 32632 ts_tablet_manager.cc:1916] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:59.872771 32632 ts_tablet_manager.cc:1929] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:23:59.873121 32632 log.cc:1199] T 205fb3f3a3844073a49da13542da2989 P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/205fb3f3a3844073a49da13542da2989
I20260709 12:23:59.874840 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 205fb3f3a3844073a49da13542da2989 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:23:59.893657 32525 raft_consensus.cc:1064] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:23:59.897382 32525 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:23:59.903793 31694 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:59.904459 31058 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:59.904666 30986 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:59.906306 32554 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:59.907131 32525 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:59.907996 32360 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:59.936391 31141 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:59.938566 32630 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:59.968464 31067 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:59.970526 30984 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:59.971392 32360 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:59.977392 30990 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:59.978101 32526 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:59.979198 32528 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:24:00.064769 32635 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.091s	user 0.004s	sys 0.000s
W20260709 12:24:00.069979 32640 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:24:00.068794 32525 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:24:00.074434 32642 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 12:24:00.073169 30986 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:00.086678 31064 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:00.106407 30887 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:00.119208 31694 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:00.122857 32624 raft_consensus.cc:1064] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:24:00.125200 32624 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:00.155144 31064 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:00.156307 31697 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 LEARNER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:00.156598 32496 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:00.156980 31141 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:00.157783 32595 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:00.160454 32595 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:24:00.181097 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 87255.2us
I20260709 12:24:00.219978 31064 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:00.228370 32526 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:00.230803 30983 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:00.238438 32496 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:00.240072 32645 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:00.260849 31694 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:00.842301 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:00.960948 31064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.002s	sys 0.008s
W20260709 12:24:00.961344 31064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.002s	sys 0.008s
I20260709 12:24:01.444082 32606 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:01.444685 32606 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.447484 32655 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.448861 32554 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.449025 32360 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.458539 32554 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.451920 31135 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.459924 32655 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.458664 32360 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.461921 32554 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.462056 32655 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.463784 32360 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.445953 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1667) took 1219 ms. Trace:
I20260709 12:24:01.464354 31136 rpcz_store.cc:276] 0709 12:24:00.226752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.226900 (+   148us) service_pool.cc:224] Handling call
0709 12:24:01.445281 (+1218381us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:01.445464 (+   183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.445630 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.445656 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.445694 (+    38us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:01.445723 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.445774 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.445874 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123668864}
I20260709 12:24:01.454749 32606 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.467473 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 182) took 1294 ms. Trace:
I20260709 12:24:01.467831 31144 rpcz_store.cc:276] 0709 12:24:00.172621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.172717 (+    96us) service_pool.cc:224] Handling call
0709 12:24:01.466181 (+1293464us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:01.467044 (+   863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.467194 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.467218 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.467255 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:01.467282 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.467331 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.467422 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23916800}
I20260709 12:24:01.471935 32554 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.472584 32554 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.478245 32525 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.478736 32525 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.480518 32525 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.480557 32360 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.480997 32525 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:01.481127 32360 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.481442 32525 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.482345 32606 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.482983 32655 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.482820 32606 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.483517 32655 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.484719 32606 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.485504 32655 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.487125 32360 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.489873 32554 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.490098 32646 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.490409 32656 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.490753 32646 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.490904 32656 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.493369 32360 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.493809 32360 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.496300 32360 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.501339 32646 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.502656 32656 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.494140 32554 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.503487 32554 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:01.504851 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.251s	user 0.004s	sys 0.010s
I20260709 12:24:01.487419 32525 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.505811 32554 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.506597 32654 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:01.507115 32654 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.509116 32654 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.518796 32654 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.519358 32654 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.524366 32559 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.524885 32559 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.523347 32652 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:01.525866 32652 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.528834 32654 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.528892 32554 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.530766 32554 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.532795 32554 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:01.535137 32651 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.183s	user 0.003s	sys 0.000s
I20260709 12:24:01.537467 32652 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.539391 32554 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.546360 32554 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.548907 32525 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:01.549623 32554 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.549815 32654 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.550390 32654 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:24:01.550792 32559 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.552227 32654 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.549551 32525 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.561847 32664 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:01.562371 32664 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.564823 32664 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.568956 32525 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.570920 32554 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.571411 32554 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.572479 30992 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:01.575423 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.575990 30985 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:01.576875 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.577740 32646 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:01.578795 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:01.580276 32648 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.580709 32648 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.581204 32648 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.581396 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.582067 30988 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
W20260709 12:24:01.580668 30986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.336s	user 0.014s	sys 0.040s
W20260709 12:24:01.583099 30986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.339s	user 0.015s	sys 0.040s
I20260709 12:24:01.584229 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1566) took 1342 ms. Trace:
I20260709 12:24:01.583014 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:01.586395 32656 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:01.583014 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.586791 32656 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.587273 30988 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:01.587283 32656 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:01.588195 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.588351 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:01.589267 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.589411 32644 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.589823 32644 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.590181 32644 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.590562 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.591179 30992 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:01.592142 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.595813 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:01.577986 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.596504 32646 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:01.596880 32646 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.597265 32646 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.597747 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.601703 32644 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.584667 30986 rpcz_store.cc:276] 0709 12:24:00.242128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.243161 (+  1033us) service_pool.cc:224] Handling call
0709 12:24:00.243478 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.243813 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.243842 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.244236 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 12:24:01.534616 (+1290380us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:01.583667 (+ 49051us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:01.583723 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.583822 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:01.583880 (+    58us) raft_consensus.cc:1685] finished
0709 12:24:01.583897 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.584141 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5221,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":3012864,"thread_start_us":1339014,"threads_started":1,"wal-append.queue_time_us":1104951}
I20260709 12:24:01.604240 32646 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.604514 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:01.605042 30986 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.606071 32644 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.606542 32646 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.607818 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1540) took 1349 ms. Trace:
I20260709 12:24:01.608173 31064 rpcz_store.cc:276] 0709 12:24:00.256979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.257112 (+   133us) service_pool.cc:224] Handling call
0709 12:24:00.257389 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.258559 (+  1170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.258614 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.259308 (+   694us) log.cc:844] Serialized 26 byte log entry
0709 12:24:00.961685 (+702377us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:00.961725 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.961796 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:01.606058 (+644262us) raft_consensus.cc:1685] finished
0709 12:24:01.606075 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.606197 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6113,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":871,"thread_start_us":701923,"threads_started":1,"wal-append.queue_time_us":1089827}
I20260709 12:24:01.611873 32645 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.612304 32655 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:24:01.613305 31694 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.348s	user 0.025s	sys 0.012s
I20260709 12:24:01.613230 32654 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.612375 32645 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.614255 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 480) took 1356 ms. Trace:
I20260709 12:24:01.614351 32644 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.614974 32644 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.615758 32645 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.614260 32654 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.616914 32644 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.614681 31694 rpcz_store.cc:276] 0709 12:24:00.259850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.260287 (+   437us) service_pool.cc:224] Handling call
0709 12:24:00.260520 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:00.260720 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:01.613718 (+1352998us) log.cc:844] Serialized 14 byte log entry
0709 12:24:01.613917 (+   199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.613963 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:01.614003 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.614067 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.614165 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:01.603662 32608 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:01.623930 32654 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.612845 32655 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.574091 32554 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.631269 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.636754 32663 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:01.637073 32654 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.637722 32654 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.644799 30986 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:01.645829 32554 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.648116 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:01.649545 30992 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.649614 32360 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.649809 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:01.650153 32360 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.661661 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.662614 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.634924 32646 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.663661 32360 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.624428 32608 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.668128 32608 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.668607 32608 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:01.669037 32608 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.670933 32608 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.671545 32608 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:01.672020 32608 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.675823 32608 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.678632 32644 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:01.678917 32655 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.679193 32644 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.637969 32663 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.646046 32559 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:01.653121 32554 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.653527 32656 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.766341 32667 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.692557 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.684434 30986 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
W20260709 12:24:01.843873 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 180248us
I20260709 12:24:01.849350 30992 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:01.855464 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.856966 30992 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:01.859854 32660 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.860401 32660 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.861675 32658 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.862179 32658 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.862639 32660 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.863147 32660 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.863592 32660 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.864605 32658 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.865561 32660 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.766803 32654 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:01.765627 32665 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.928119 32670 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:01.876346 32659 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:01.928681 32670 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.941252 32670 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:01.792296 32667 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:01.912554 32665 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.954237 32665 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:01.984056 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.984970 30992 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 1.
I20260709 12:24:01.985633 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.986001 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.996281 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1666) took 1773 ms. Trace:
I20260709 12:24:01.996675 31139 rpcz_store.cc:276] 0709 12:24:00.222924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.223080 (+   156us) service_pool.cc:224] Handling call
0709 12:24:01.973831 (+1750751us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:01.974020 (+   189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.974205 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.995980 (+ 21775us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.996024 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:01.996072 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.996129 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.996228 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99797248}
I20260709 12:24:02.027956 30986 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:02.032589 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:02.033325 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:01.943816 32663 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.038830 32666 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.014591 32670 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:01.939797 32659 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.043382 32670 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.045996 32659 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.046453 32670 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.046629 30984 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:02.035880 32664 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:02.050024 32677 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.050407 32670 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.050938 32670 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.050732 32677 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.053411 32670 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.042804 32663 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.066277 32667 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.075686 32662 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.087930 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:02.084791 32656 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:02.090282 32680 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:24:02.091853 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 60689.4us
I20260709 12:24:02.054431 32664 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.080036 32659 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.084722 32670 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.081415 32667 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.053622 32677 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.076365 32662 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.107823 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:02.086539 32668 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:02.073222 32663 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.112655 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:02.115024 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.094781 32665 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.135751 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:02.137495 30992 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:02.136055 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:02.138615 30986 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.140838 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:02.090811 32680 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.149856 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:02.179710 32683 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.182307 32683 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.049258 32666 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.180750 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:02.104185 32659 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.111320 32667 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.188361 32496 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.124743 32670 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.200037 32496 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.202310 32661 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.146749 32665 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.151297 32668 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.212278 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:02.212805 32676 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.213114 30991 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:02.215768 32676 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.202931 32661 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.232295 32669 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.203410 32675 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.234345 32669 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.237191 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:02.242383 32675 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.259132 32656 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:02.260120 32682 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.260290 32656 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:02.260684 32656 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:02.291911 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 140084us
I20260709 12:24:02.293635 32496 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.297858 32526 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:02.299072 32526 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:02.299997 32680 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:02.300486 32680 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.300902 32680 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.306306 32526 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.298625 32683 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.151369 32655 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.260628 32682 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.336787 32662 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:02.338116 32681 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.338343 32680 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.345301 32655 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:02.345953 32669 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.345883 32663 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:02.346278 32655 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:02.370918 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 291178us
I20260709 12:24:02.401281 30986 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:02.405508 32686 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.406100 32686 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.526341 32687 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.595110 32690 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.595741 32690 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.595281 32668 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.600797 32526 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.527004 32687 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.338801 32681 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.648707 32526 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:02.600539 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 307616us
I20260709 12:24:02.649863 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:02.656693 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:02.659049 32691 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:02.659588 32691 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:02.660090 32691 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.587564 32659 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:02.603060 32664 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.602515 32683 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.621798 32669 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.665985 32669 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.667719 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:02.649766 32666 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:02.667984 32669 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.622100 32680 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.621981 32662 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:02.665283 32683 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.670392 31690 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:02.603025 32675 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:02.665383 32684 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.668757 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:02.673481 31689 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:02.673622 32687 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.674331 32678 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.675554 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:02.674392 32683 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:02.691720 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.676662 32678 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.693423 32692 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.696666 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:02.697335 30987 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.683152 32687 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.694011 32692 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.675273 32680 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.675155 32662 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.720605 32655 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:02.721887 32692 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:02.735051 32692 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.723589 32681 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.735541 32692 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.738345 32688 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:02.741909 32688 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:02.742018 32692 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.742690 32692 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:02.743003 32692 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:02.743359 32692 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:02.728899 32689 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.380s	user 0.001s	sys 0.000s
I20260709 12:24:02.721163 32655 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:02.752110 32655 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.752614 32699 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.753181 32699 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.755251 32699 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:02.703282 32684 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.759061 32662 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:02.757877 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 65185us
I20260709 12:24:02.761356 32656 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.757719 32684 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:02.762300 32655 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.763559 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1276) took 2600 ms. Trace:
I20260709 12:24:02.768272 31141 rpcz_store.cc:276] 0709 12:24:00.162713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.163561 (+   848us) service_pool.cc:224] Handling call
0709 12:24:02.303313 (+2139752us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876feee 000000000036eb39 00007f2ef07e31e0 00007f2ef08ac0c4 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:02.303448 (+   135us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.303776 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.303801 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.339789 (+ 35988us) log.cc:844] Serialized 556 byte log entry
0709 12:24:02.343518 (+  3729us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:02.343575 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.343670 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.747170 (+403500us) raft_consensus.cc:1685] finished
0709 12:24:02.747198 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.763475 (+ 16277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151082752,"thread_start_us":3371,"threads_started":1,"wal-append.queue_time_us":3692}
I20260709 12:24:02.728161 32696 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:02.705695 32687 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.776010 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:02.776655 31067 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:02.770148 32656 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:02.769915 32690 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:02.743101 32688 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:02.769783 32696 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:02.778075 32680 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:02.818015 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.011529 30987 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:03.012852 32697 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:02.774112 32678 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:02.770962 31090 debug-util.cc:398] Leaking SignalData structure 0x7b080016d760 after lost signal to thread 29845
I20260709 12:24:03.064028 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.068235 31067 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:03.115253 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.013846 32655 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:03.154778 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 380138us
W20260709 12:24:03.062460 31090 debug-util.cc:398] Leaking SignalData structure 0x7b080016d780 after lost signal to thread 29848
I20260709 12:24:02.829420 32687 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.013473 32697 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.157361 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:02.810833 32694 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:03.112813 32675 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:03.159696 30986 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.160612 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:03.162448 30987 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.163949 32696 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.166266 32692 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:01.661244 32525 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:03.168207 32692 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.171762 30986 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:03.173082 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.170763 32525 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:03.174373 32525 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.171042 32675 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.172207 32694 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.181690 30987 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
W20260709 12:24:03.178191 32691 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.004s
I20260709 12:24:03.184350 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.185400 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.185930 31067 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:03.188779 32697 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.189040 32675 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.189632 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.190270 31067 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
W20260709 12:24:03.193519 32648 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.612s	user 0.020s	sys 0.010s
I20260709 12:24:03.185001 32691 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.196115 32691 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:03.194026 32694 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.197702 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.198089 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 547) took 1104 ms. Trace:
I20260709 12:24:03.198444 31694 rpcz_store.cc:276] 0709 12:24:02.093872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.197270 (+1103398us) service_pool.cc:224] Handling call
0709 12:24:03.197587 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.197755 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.197781 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.197821 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:03.197851 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.197899 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.197994 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.199282 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.199889 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 187) took 1102 ms. Trace:
I20260709 12:24:03.200078 31136 rpcz_store.cc:276] 0709 12:24:02.097711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.199054 (+1101343us) service_pool.cc:224] Handling call
0709 12:24:03.199823 (+   769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.202188 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.193894 32648 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.203898 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.205286 32648 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:03.207744 32468 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:03.208302 32468 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.209367 32694 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:03.209966 32468 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.211705 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.211659 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.213052 31067 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:03.212968 32694 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.212308 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.214015 31058 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:03.214803 32694 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.211964 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.216609 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.217180 31063 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:03.217849 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.218147 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.218962 32692 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:03.219116 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.219312 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.220503 31063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "205fb3f3a3844073a49da13542da2989"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:24:03.220815 31060 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:24:03.221033 31063 raft_consensus.cc:606] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:03.219453 32692 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.221611 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.223413 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.224233 32692 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.224623 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.226249 31063 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:01.684723 32646 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.228706 32646 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:03.224689 32692 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:03.229386 32692 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.230327 31060 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:03.232609 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.233649 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.234099 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 552) took 1083 ms. Trace:
I20260709 12:24:03.234459 31696 rpcz_store.cc:276] 0709 12:24:02.150923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.233347 (+1082424us) service_pool.cc:224] Handling call
0709 12:24:03.233601 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.233735 (+   134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.233757 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.233791 (+    34us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:03.233819 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.233865 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.233943 (+    78us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.234911 31062 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:03.235260 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:03.232774 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.693s	user 0.028s	sys 0.065s
W20260709 12:24:03.236465 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.697s	user 0.029s	sys 0.065s
W20260709 12:24:03.237942 32645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.551s	user 0.034s	sys 0.068s
W20260709 12:24:03.238533 32645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.554s	user 0.035s	sys 0.068s
I20260709 12:24:03.239988 32680 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:03.240469 32680 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.236806 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.243376 32680 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:24:03.244400 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.244627 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 553) took 1093 ms. Trace:
W20260709 12:24:03.245324 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.245954 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:03.246639 32692 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.247360 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.248492 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.249025 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.249910 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.250500 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.250726 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.251376 32646 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.252202 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:03.268481 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.261936 32680 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:03.267189 32709 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:03.269629 32680 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.270752 32709 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.271318 32525 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:03.272014 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:03.272604 32525 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:03.272691 31136 raft_consensus.cc:606] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:03.273553 32525 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.262857 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:03.262058 32646 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:03.275910 32688 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.245054 31692 rpcz_store.cc:276] 0709 12:24:02.151422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.243799 (+1092377us) service_pool.cc:224] Handling call
0709 12:24:03.244102 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.244264 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.244297 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.244344 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:03.244386 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.244453 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.244574 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.278412 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:03.280117 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.281111 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.281378 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.331887 30991 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:03.284257 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.333401 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.338506 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:03.377746 30987 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.333038 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.320865 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.375908 31061 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:03.321924 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.287477 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:03.384797 30992 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.385499 31136 raft_consensus.cc:606] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:03.388857 31062 raft_consensus.cc:2393] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:03.332183 31106 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.66:45157 timed out after 1.609s (SENT)
I20260709 12:24:03.392704 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.394178 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1567) took 1852 ms. Trace:
I20260709 12:24:03.396775 30983 rpcz_store.cc:276] 0709 12:24:01.541322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.393476 (+1852154us) service_pool.cc:224] Handling call
0709 12:24:03.393741 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.393887 (+   146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.393913 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.393950 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:03.393977 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.394024 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.394128 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.357664 32652 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:03.330673 32680 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.360241 32675 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:03.336351 32644 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.475374 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.330834 32688 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:03.476220 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:03.476284 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 87675.9us
I20260709 12:24:03.477092 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.478281 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.480490 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.488284 30983 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
W20260709 12:24:03.481256 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 90201.5us
I20260709 12:24:03.489777 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.490453 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.491466 32688 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.493230 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.493945 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.494674 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.343331 32646 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.495643 31689 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:03.496907 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.474352 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.479970 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.495746 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.494765 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.495115 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.479717 30992 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:03.504093 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.504254 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.504844 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.498035 31696 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:03.505376 30983 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:03.506314 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:03.506445 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.506815 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.507031 31694 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:03.507947 32526 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:03.508350 32526 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:03.508677 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.510259 32526 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.509194 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.512979 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:03.515354 30958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.263s	user 0.017s	sys 0.023s
W20260709 12:24:03.515669 30958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.263s	user 0.017s	sys 0.023s
I20260709 12:24:03.517724 30987 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:03.524461 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.527066 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.527549 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.527884 32714 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:03.528173 31692 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:03.529660 32554 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:03.497740 32559 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.546787 32559 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.546973 32713 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:03.547503 32713 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:03.548405 32713 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.508129 32707 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:24:03.482735 32652 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.549209 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.571435 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.571933 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:03.572829 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 240765us
I20260709 12:24:03.574659 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.569977 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.583004 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.584257 31063 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:03.584555 31063 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:03.584766 32652 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.586203 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.502949 32675 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.577186 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.585296 31063 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:03.596122 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 73474.7us
I20260709 12:24:03.600311 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.601426 30992 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:03.570609 32665 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.602895 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.594784 32559 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.606038 32559 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.617293 31063 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.620929 32675 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.622325 32652 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:03.622458 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.622835 32652 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.623854 32718 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:03.624351 32718 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.624832 32652 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.627465 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.628470 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.628870 32606 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:03.629257 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:03.630856 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.631877 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:03.632830 32700 raft_consensus.cc:993] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:24:03.633777 32700 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.635037 32654 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:03.636261 32654 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:03.636715 32654 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.636979 32559 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.629482 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.635572 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:03.640497 32688 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:03.641181 32688 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:03.635174 32554 raft_consensus.cc:1081] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:24:03.646319 32700 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:24:03.649824 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1568) took 2103 ms (client timeout 2103 ms). Trace:
I20260709 12:24:03.665517 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.653216 32688 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:03.665931 30984 rpcz_store.cc:269] 0709 12:24:01.546101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.597524 (+ 51423us) service_pool.cc:224] Handling call
0709 12:24:03.649623 (+2052099us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b540022d8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:03.649717 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030997760}
I20260709 12:24:03.654343 32686 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.661775 32661 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.664052 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 187) took 1680 ms. Trace:
I20260709 12:24:03.662851 32646 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:24:03.678566 30988 rpcz_store.cc:276] 0709 12:24:01.983318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.983468 (+   150us) service_pool.cc:224] Handling call
0709 12:24:01.997849 (+ 14381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.531337 (+1533488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.531368 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.531404 (+    36us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:03.531431 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.663853 (+132422us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:03.663890 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.663998 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3526304000}
I20260709 12:24:03.635972 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:03.665258 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 150862us
I20260709 12:24:03.620234 32526 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:03.674857 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.627166 32718 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:03.683292 30990 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1569) took 2117 ms (client timeout 2042 ms). Trace:
W20260709 12:24:03.683696 30990 rpcz_store.cc:269] 0709 12:24:01.570920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.676443 (+2105523us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:03.676555 (+   112us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:03.684768 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 550) took 1576 ms. Trace:
I20260709 12:24:03.685144 31695 rpcz_store.cc:276] 0709 12:24:02.108897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.684013 (+1575116us) service_pool.cc:224] Handling call
0709 12:24:03.684286 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.684449 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.684478 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.684517 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:03.684549 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.684614 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.684712 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.634184 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:03.688160 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54076us
I20260709 12:24:03.690785 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.691386 31144 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:03.696506 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:03.702946 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:03.703485 30991 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.704183 31063 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:03.705863 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.706478 30990 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:03.692353 32646 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:03.714890 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.692531 32667 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.707073 31106 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:03.676539 32700 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.716900 32645 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:03.717756 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 131477us
I20260709 12:24:03.720281 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.720604 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:03.722187 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:03.708622 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.709420 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800286860 after lost signal to thread 29848
I20260709 12:24:03.727353 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:03.728052 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800326a80 after lost signal to thread 30924
W20260709 12:24:03.729087 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800034c40 after lost signal to thread 31016
W20260709 12:24:03.729883 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08000343e0 after lost signal to thread 31093
W20260709 12:24:03.730871 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08002185a0 after lost signal to thread 31168
I20260709 12:24:03.709862 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:03.731881 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08002258c0 after lost signal to thread 31722
I20260709 12:24:03.706197 32554 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:03.711264 31144 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:03.712387 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.712687 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.735078 31141 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:03.715374 32646 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:03.751602 31064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:24:03.752199 31064 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:03.752550 31064 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:03.738296 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:03.726876 32644 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:03.760039 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.775393 31068 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:03.775741 31068 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:03.736164 32554 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:03.751736 32646 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.751127 32525 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:03.775530 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:03.778720 31068 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.758293 32652 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:03.817575 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 100600us
I20260709 12:24:03.818753 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:03.817206 31068 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:03.819746 31140 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:03.776714 32703 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:03.820803 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.778100 32554 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.826591 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.779901 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:03.755716 32695 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:24:03.869787 31064 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:03.827160 31065 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:03.817471 32662 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:03.827811 32655 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:03.872463 32724 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:24:03.915766 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.917059 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.917366 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.913686 32662 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:03.930955 32662 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:03.874006 30988 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:03.833981 32682 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.870572 32652 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.823266 32703 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:03.880345 32676 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:03.884241 31068 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:03.885008 32654 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:03.876873 30984 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:03.918148 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.918421 30987 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:03.923074 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.926909 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.927248 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.927984 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.929769 31137 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:03.962563 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.931828 31064 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:03.934178 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 548) took 1840 ms. Trace:
I20260709 12:24:03.934094 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.966249 31142 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:03.947872 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.933024 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:03.970096 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:03.959703 31141 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:03.965312 31697 rpcz_store.cc:276] 0709 12:24:02.094269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.932046 (+1837777us) service_pool.cc:224] Handling call
0709 12:24:03.933722 (+  1676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.933899 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.933924 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.933965 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:03.933992 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.934044 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.934130 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.987632 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:03.988047 31135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 186) took 1893 ms (client timeout 1743 ms). Trace:
W20260709 12:24:03.988430 31135 rpcz_store.cc:269] 0709 12:24:02.094518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.987894 (+1893376us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:03.987995 (+   101us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:03.989181 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:03.989647 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.989851 31143 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:03.991148 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.992476 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:03.946667 32703 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:03.928824 32720 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:03.978531 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:04.015100 31139 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.015548 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:04.013588 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.016461 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:04.016954 31068 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:04.018689 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:04.018905 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 130002us
I20260709 12:24:04.019301 31697 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:04.019536 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:04.019968 30991 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:04.020608 30992 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:04.020699 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:04.015074 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.021358 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:04.021657 31690 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:04.021900 31698 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.022444 32730 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.022977 32730 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:04.023315 32730 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.024582 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.025279 32652 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.017800 32720 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.029527 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:04.030256 30991 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.031711 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.031879 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.034296 31065 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:04.018436 32654 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.034631 31065 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:04.046700 32720 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.048038 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.050971 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.052321 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.053959 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.041110 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.057145 31698 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:04.061292 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.065565 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1575) took 2464 ms. Trace:
I20260709 12:24:04.074041 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:04.074345 30989 rpcz_store.cc:276] 0709 12:24:01.572807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.036828 (+2464021us) service_pool.cc:224] Handling call
0709 12:24:04.037089 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.037229 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.037252 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.037285 (+    33us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:04.037310 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.037353 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.037435 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:04.042979 32601 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.069342 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.072352 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.073140 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.031200 32729 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:04.076046 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.076076 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.083849 32700 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:04.096526 30988 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:04.097532 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:04.099126 31691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 554) took 1764 ms (client timeout 1978 ms). Trace:
W20260709 12:24:04.099490 31691 rpcz_store.cc:269] 0709 12:24:02.335089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.095995 (+1760906us) service_pool.cc:224] Handling call
0709 12:24:04.099044 (+  3049us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:04.101836 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 71245.3us
I20260709 12:24:04.094087 31698 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:04.063689 31065 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.103188 31065 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.102470 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.108682 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.109539 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:24:04.113672 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54346.2us
I20260709 12:24:04.094579 32729 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:04.116010 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:04.118808 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:04.104728 32700 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:04.119892 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.120900 31065 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:04.110901 32645 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:24:04.120565 32630 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.122570 32630 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:04.122967 32630 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.124658 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:04.126431 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.121315 32665 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:04.129208 32665 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:04.129626 32665 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:04.129361 32700 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.128181 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:04.131381 32731 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.131075 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:04.120237 32729 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:04.126569 31090 debug-util.cc:398] Leaking SignalData structure 0x7b0800130e60 after lost signal to thread 30924
I20260709 12:24:04.137065 32673 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.137478 32673 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:04.137812 32673 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.134135 32678 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.140290 32678 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:04.127513 32601 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.140657 32678 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.142098 32601 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:04.147768 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1591) took 2561 ms. Trace:
I20260709 12:24:04.121555 32692 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:24:04.127899 32682 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.148157 30985 rpcz_store.cc:276] 0709 12:24:01.580786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.598938 (+ 18152us) service_pool.cc:224] Handling call
0709 12:24:01.599863 (+   925us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.600174 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.600199 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.600946 (+   747us) log.cc:844] Serialized 494 byte log entry
0709 12:24:01.601085 (+   139us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:01.601115 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.601165 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.142173 (+2541008us) raft_consensus.cc:1685] finished
0709 12:24:04.142193 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.142344 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:04.125689 32554 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.125325 32676 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:24:04.152259 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:04.149245 31090 debug-util.cc:398] Leaking SignalData structure 0x7b08000814a0 after lost signal to thread 30958
I20260709 12:24:04.149801 32682 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:04.148103 32601 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.158962 31142 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 3: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:04.155134 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:04.157126 32554 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:04.172673 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.172816 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.173341 30985 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:04.158608 32682 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.175401 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:04.162082 32661 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:04.179088 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.179111 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.159722 32692 consensus_queue.cc:1243] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:04.160163 32703 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.175971 32667 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.189348 32671 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:04.195851 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.182705 32701 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:04.200457 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.180693 32661 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:04.203843 30987 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:04.205819 31140 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:04.206579 32526 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:04.207479 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.207695 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 551) took 2097 ms. Trace:
I20260709 12:24:04.156491 32655 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:04.197924 32667 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:04.209173 31693 rpcz_store.cc:276] 0709 12:24:02.110600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.174934 (+2064334us) service_pool.cc:224] Handling call
0709 12:24:04.205224 (+ 30290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.207370 (+  2146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.207404 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.207444 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:04.207481 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.207539 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.207646 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:04.217141 32662 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.218469 32661 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.219295 32703 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:04.208271 32701 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:04.222837 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.212625 32667 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.204979 30991 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:04.226110 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.209406 31059 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:04.227439 31139 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:04.228487 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:04.231573 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 119259us
I20260709 12:24:04.232400 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.233882 32732 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:04.235237 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.235947 30985 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.211665 32646 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.243432 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:04.244235 31139 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:04.244603 31139 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:04.245481 31139 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:04.248142 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 65413.8us
I20260709 12:24:04.249367 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.249769 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.250813 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:04.251120 32676 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:04.251952 32676 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.255651 31135 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 3: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:04.256433 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.257709 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.252368 32676 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:04.263413 30985 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.267179 30988 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:04.269784 32739 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.229559 32701 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.274328 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.274986 30988 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.277654 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.278667 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.263444 32630 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.280221 32732 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.281446 32630 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:04.265877 32676 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.290648 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.286146 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.295063 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.300891 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.306314 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.319793 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.346203 31695 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:04.348919 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.350010 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:04.350135 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 44046us
I20260709 12:24:04.351603 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.352854 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.350951 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:04.283509 32740 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.352231 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:04.307898 31140 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.355331 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:04.290318 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.303818 32645 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:04.303179 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.303560 32707 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:04.376776 31140 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:04.351017 32739 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:04.354377 32662 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:04.356715 32675 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:04.558981 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 203690us
I20260709 12:24:04.422513 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.289949 32678 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:04.484323 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.559898 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.560060 31140 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.565109 31141 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:04.564615 30958 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:04.571816 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.572813 30984 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:04.564328 31696 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:04.574368 31140 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:04.578547 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.369508 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:04.595373 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 187712us
I20260709 12:24:04.597360 32675 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:04.541808 31691 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:04.601482 31136 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:04.573539 32660 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:04.603510 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.603336 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.630194 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:04.634011 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 176044us
I20260709 12:24:04.582386 32665 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.627449 32646 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.631145 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.638720 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.641477 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 1573) took 1415 ms. Trace:
I20260709 12:24:04.641984 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:04.644918 30988 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
W20260709 12:24:04.638415 32736 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:04.561131 32739 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.653632 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.641842 31060 rpcz_store.cc:276] 0709 12:24:03.223422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.224401 (+   979us) service_pool.cc:224] Handling call
0709 12:24:04.638913 (+1414512us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b54002cf1d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:04.639028 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11538688}
I20260709 12:24:04.653789 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:04.660267 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 289522us
I20260709 12:24:04.661518 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:04.664232 32678 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:04.665781 30990 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:04.667483 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.679656 31696 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:04.681224 32682 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.683290 32682 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:04.686789 32730 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.000s
I20260709 12:24:04.687278 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.691308 32665 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:04.691440 32713 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.687207 32730 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.681023 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.696424 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.698104 31140 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:04.696002 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:04.698300 31064 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:04.704617 32750 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:04.705448 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:04.705701 32750 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:04.706073 32750 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:04.706686 32739 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:04.706951 32750 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.708039 32682 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:04.708195 32750 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:04.707948 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.660593 32714 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:04.707917 32525 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:04.692123 32673 consensus_meta.cc:287] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.002s
I20260709 12:24:04.711834 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.714025 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.708899 32676 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.714272 31136 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:04.710528 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.710783 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.716672 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.718657 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:04.719547 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.700842 32678 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:04.721239 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.721993 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.713107 32525 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:04.724367 32678 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.729142 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.720498 31690 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:04.725102 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 63642us
I20260709 12:24:04.716513 32655 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45358 (request call id 85) took 4492 ms. Trace:
I20260709 12:24:04.727715 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:04.725970 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.708899 32682 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:04.722388 32729 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.581s	user 0.005s	sys 0.005s
I20260709 12:24:04.733775 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.724522 32525 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.713013 32714 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.736301 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:04.737164 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:04.737622 32730 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:04.737696 32676 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.738431 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.743113 32678 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.733284 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:04.746402 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.716840 32673 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.746646 31140 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.748525 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.749990 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.753880 32758 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:04.750681 32700 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.763998 32676 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:04.764171 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.764312 32714 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:04.765004 32678 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:04.765143 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.765784 31137 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:04.767197 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:04.767807 32665 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.768157 32665 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:04.768692 32665 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.770736 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:04.768163 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.773192 31064 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:04.745793 32729 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.773027 31033 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.773912 32676 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.775458 31691 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.776777 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:04.772809 32601 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.001s
I20260709 12:24:04.782199 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.776109 32676 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
W20260709 12:24:04.813192 31090 debug-util.cc:398] Leaking SignalData structure 0x7b0800353140 after lost signal to thread 31016
I20260709 12:24:04.774842 32678 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:04.814185 32678 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:04.731894 32655 rpcz_store.cc:276] 0709 12:24:00.224240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.226070 (+  1830us) service_pool.cc:224] Handling call
0709 12:24:01.450182 (+1224112us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086e1c0 00007f2ef085a9d9 00007f2ef085a442 00007f2ee9dd6c54
0709 12:24:01.463423 (+ 13241us) log.cc:844] Serialized 494 byte log entry
0709 12:24:04.716285 (+3252862us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5092,"raft.run_cpu_time_us":1268,"raft.run_wall_time_us":1268,"spinlock_wait_cycles":73560320}
I20260709 12:24:04.827319 32645 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.827733 32645 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:24:04.809841 32667 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.001s
I20260709 12:24:04.812646 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:04.831911 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 49770.1us
I20260709 12:24:04.832231 31136 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:04.833488 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:04.833619 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.811874 32700 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.834187 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.839099 30992 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:04.839103 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.838994 32727 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:04.834909 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d50d1465e4b84136bf90de700a93ef2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:04.835705 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.772512 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.828083 32645 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.856015 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:04.858002 30986 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:04.809562 32741 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 12:24:04.850593 31136 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:04.846980 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.866400 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.841396 32667 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.868314 32667 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.869179 32682 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.869586 32682 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:04.869817 32682 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:04.870450 32682 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.874207 32673 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:04.862342 32729 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:04.888124 30986 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:04.887231 32709 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:04.862874 31090 debug-util.cc:398] Leaking SignalData structure 0x7b08000813c0 after lost signal to thread 31093
I20260709 12:24:04.891991 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:04.892591 31068 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 1.
W20260709 12:24:04.896097 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 111008us
I20260709 12:24:04.902490 31138 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.902678 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.903507 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.904273 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.904625 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.824793 32601 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.911099 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.911744 32601 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:04.911928 31143 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 1.
I20260709 12:24:04.912535 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:04.913133 31694 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:04.884096 32727 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:04.913820 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.919170 32727 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:04.922155 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.922797 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:04.923590 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.905916 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:04.932058 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 64348.5us
I20260709 12:24:04.932837 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.937532 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.933740 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:04.933616 32655 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:04.934674 32755 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:04.934859 32682 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.936009 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:04.921183 32652 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.931047 32720 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:04.947307 31090 debug-util.cc:398] Leaking SignalData structure 0x7b0800300380 after lost signal to thread 31168
W20260709 12:24:04.953356 31090 debug-util.cc:398] Leaking SignalData structure 0x7b08000ff820 after lost signal to thread 31169
I20260709 12:24:04.940307 32762 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:24:04.958791 32762 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:04.959125 32762 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:04.945200 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.941676 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:04.963287 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.948009 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:04.949749 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:04.951035 32749 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.951643 32761 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:24:04.965534 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:04.968314 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.976123 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:04.977207 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.942088 32714 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.961000 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.981235 32652 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:04.981693 32709 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.982735 32645 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.983577 32741 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.972412 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.984839 32645 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:04.972411 32744 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.970038 32720 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:04.973358 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.970695 32671 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:05.002795 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:04.977994 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.004225 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.004354 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.005108 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:04.980108 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 47245.3us
I20260709 12:24:04.986286 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.006068 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:04.970872 32740 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:05.007090 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.007426 32740 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.931434 32725 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:04.976692 31090 debug-util.cc:398] Leaking SignalData structure 0x7b0800243cc0 after lost signal to thread 31722
I20260709 12:24:05.009375 32725 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:04.989408 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.978477 32608 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:04.990172 32721 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.986308 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.012071 32716 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.012316 31064 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.012476 32716 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:05.012729 31064 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:05.012943 32716 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.991740 32665 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:04.996603 30983 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:04.986303 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 484) took 1380 ms. Trace:
I20260709 12:24:04.998831 31106 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.000556 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1686) took 1031 ms. Trace:
I20260709 12:24:05.035909 31135 rpcz_store.cc:276] 0709 12:24:03.969755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.047730 (+ 77975us) service_pool.cc:224] Handling call
0709 12:24:05.000375 (+952645us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b540029efd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:05.000501 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475427200}
I20260709 12:24:05.001950 31058 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.002754 31063 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.037248 31058 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:05.037357 31063 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:04.946838 32684 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.008195 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:05.009084 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.014391 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.013515 31064 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.013293 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:04.987742 32720 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:05.025404 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:05.094715 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 69510.7us
I20260709 12:24:05.094889 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:05.095352 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 70053.8us
I20260709 12:24:05.095947 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:05.096040 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:04.976799 32730 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:05.055388 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:05.096961 31065 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.007462 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.097209 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.008450 32660 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:05.015372 32671 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:05.101361 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.100309 31058 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.015092 32761 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.117587 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.109014 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:05.110291 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:05.119328 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:05.097563 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.107354 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:05.117494 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 118773us
I20260709 12:24:05.120453 31058 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.034636 32721 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:05.120975 31106 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.122370 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.123315 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:05.124886 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.125924 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.127707 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:05.128571 32691 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.056510 32684 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:05.129482 32684 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.131959 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.133342 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:05.135172 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1638) took 1240 ms. Trace:
I20260709 12:24:05.135522 30983 rpcz_store.cc:276] 0709 12:24:03.894349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.969011 (+ 74662us) service_pool.cc:224] Handling call
0709 12:24:04.630016 (+661005us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:05.135037 (+505021us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b5400225454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:05.135123 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914809344}
I20260709 12:24:05.137217 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:05.138667 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:05.104638 32608 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.098762 32659 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.145243 31140 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.098964 32720 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.119746 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.149663 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.155457 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1697) took 1093 ms. Trace:
I20260709 12:24:05.106546 32741 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.155824 31142 rpcz_store.cc:276] 0709 12:24:04.051548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.052062 (+   514us) service_pool.cc:224] Handling call
0709 12:24:05.144835 (+1092773us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540006f9d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:05.145022 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240285952}
I20260709 12:24:05.102397 31143 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:05.015552 32525 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.165424 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.158829 31058 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:05.175028 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55366.1us
I20260709 12:24:05.149945 32655 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.114369 32630 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 12:24:05.187664 31136 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.147670 32721 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.159814 32554 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:24:05.188405 32608 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.202473 32601 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.120172 32656 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:24:05.205662 30887 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 311) took 1500 ms. Trace:
I20260709 12:24:05.205997 30887 rpcz_store.cc:276] 0709 12:24:03.705672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.705791 (+   119us) service_pool.cc:224] Handling call
0709 12:24:05.205576 (+1499785us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:24:05.206609 32630 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:05.207010 32630 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:05.154323 32644 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.207726 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.208540 31144 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 3: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:05.208932 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1684) took 1240 ms. Trace:
I20260709 12:24:05.209254 31144 rpcz_store.cc:276] 0709 12:24:03.968252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.968383 (+   131us) service_pool.cc:224] Handling call
0709 12:24:05.207586 (+1239203us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:05.208881 (+  1295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2785664}
I20260709 12:24:05.210991 30889 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 2 to 3, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:05.118887 32707 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 12:24:05.035602 31692 rpcz_store.cc:276] 0709 12:24:03.605605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.990105 (+384500us) service_pool.cc:224] Handling call
0709 12:24:04.949424 (+959319us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fe9c 00007f2ee876fa5b 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:04.973894 (+ 24470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.985909 (+ 12015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.985939 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.985990 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:04.986024 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.986083 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.986197 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24478848}
I20260709 12:24:05.154430 32744 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.154167 32739 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.197669 31141 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.094600 32762 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.186981 32761 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:05.149786 32740 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.225056 31696 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:05.117441 32676 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:24:05.194276 32765 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:05.097935 32755 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
W20260709 12:24:05.215294 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 106914us
I20260709 12:24:05.161379 32741 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:05.198341 32525 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:05.222652 32750 consensus_meta.cc:287] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.000s
I20260709 12:24:05.227023 32671 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.250649 32743 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:24:05.256659 32665 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.258081 32648 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:24:05.272310 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:05.267185 32755 consensus_queue.cc:1243] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:05.275573 32729 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.275960 32729 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:05.276371 32743 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.276710 32743 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:05.227559 32761 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:05.258464 32741 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.277287 32684 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.266588 32676 consensus_queue.cc:1243] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:05.285688 32714 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:05.207743 32630 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.282390 32761 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.265763   303 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:24:05.277571 32743 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.292153   303 consensus_queue.cc:1243] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:24:05.292354 32767 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:24:05.292587 32743 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.315335 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:05.187897 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:05.319634 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 195519us
W20260709 12:24:05.321007 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 133088us
I20260709 12:24:05.321240 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.317993 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:05.325686 31061 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:05.317670 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 125840us
I20260709 12:24:05.326526 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.327921 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.335418 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.337877 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.339087 30989 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:05.344282 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.342149 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:05.351712 32630 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:05.353574 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 100932us
I20260709 12:24:05.359727 32659 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.361356 32559 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:05.361846 32559 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:05.363233 32654 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.364037 32525 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.363914 32659 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:05.369792 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:05.373080 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:05.374084 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1389) took 2091 ms. Trace:
I20260709 12:24:05.369979 32661 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.375497 32665 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.329748 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.385061 32559 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.403949 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 575) took 1067 ms. Trace:
I20260709 12:24:05.385313 32654 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:05.276739 32729 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:05.412884 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 68871.6us
I20260709 12:24:05.416575 32684 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.418025 32682 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:05.406664 32654 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.419183 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.422315 32525 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.422444   304 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.422555 31696 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:05.423961 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:05.425074 32729 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.393953 31067 rpcz_store.cc:276] 0709 12:24:03.282641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.498427 (+215786us) service_pool.cc:224] Handling call
0709 12:24:04.639210 (+1140783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.639390 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.639413 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.639451 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:04.639468 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.639501 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.373554 (+734053us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002cf1d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:05.373787 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622464896}
I20260709 12:24:05.404390 31698 rpcz_store.cc:276] 0709 12:24:04.290211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.296002 (+  5791us) service_pool.cc:224] Handling call
0709 12:24:04.296357 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.372194 (+ 75837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.372213 (+    19us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.372241 (+    28us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:04.372259 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.357728 (+985469us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b5400514d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:05.357837 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.358081 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165262080}
I20260709 12:24:05.430030 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:05.433876 32676 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:05.425683 31063 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.440557 30984 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.441408 32659 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:05.442268 32659 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:05.444208 30990 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:05.443729 32659 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.432688   304 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.447722 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:05.448043 32670 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.451721 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.453989 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:05.454566 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 125052us
I20260709 12:24:05.455533   304 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:05.456243   304 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.479849 31063 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.483912 32659 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.487296 32725 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.484124 31137 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:05.489423   302 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:05.499289 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.496872 32725 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:05.502381 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 85042.9us
W20260709 12:24:05.510382 32747 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.868s	user 0.007s	sys 0.005s
I20260709 12:24:05.511847 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:05.517827 32762 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.484045 32716 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.521622 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:05.522608 32749 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.509717 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:05.534446 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.538564 31067 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:05.542704 31067 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.543919 32360 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:05.499234 32654 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.546344 32749 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:05.549604 31063 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.553156 31059 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:05.557708 32754 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.531387 32761 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.562706 31061 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:05.553032 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:05.559554 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.551687   303 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.564569 32761 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.576365 30983 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.570748 31059 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:05.558459 32754 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.580108 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 582) took 1258 ms. Trace:
W20260709 12:24:05.581741 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.582779 32756 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.583475 32654 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:05.578773 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:05.592558 32701 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:05.594071 32714 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:05.594262 32727 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:05.596500 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:05.596712 32714 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:05.586943 31695 rpcz_store.cc:276] 0709 12:24:04.319145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.361640 (+ 42495us) service_pool.cc:224] Handling call
0709 12:24:05.377362 (+1015722us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:05.377745 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:05.378956 (+  1211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.378988 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:05.400488 (+ 21500us) log.cc:844] Serialized 494 byte log entry
0709 12:24:05.561715 (+161227us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:05.561751 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.561814 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:05.564523 (+  2709us) raft_consensus.cc:1685] finished
0709 12:24:05.564555 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.577430 (+ 12875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42474752,"thread_start_us":142382,"threads_started":1,"wal-append.queue_time_us":154163}
W20260709 12:24:05.597880 32686 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:24:05.597038 32727 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.600821 32695 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.605208 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:05.611840 32692 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:24:05.597155 32714 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:05.605571 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73235.4us
I20260709 12:24:05.620589 32714 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.621173 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.621992 32692 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.613564 32559 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.624560 31064 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.624889 32559 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.605036 32721 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.622735 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:05.621999 32714 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.628538 32716 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.628876 31067 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
W20260709 12:24:05.631415   311 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:24:05.630085 32754 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.641062 31062 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.618706 32695 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:05.633587 31058 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
W20260709 12:24:05.646451 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54822us
I20260709 12:24:05.618849 32727 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:05.641266 32630 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.652262 32630 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.654201 32496 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:05.654454 32630 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.640784 31139 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.654573 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:05.654758 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.655263 31062 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:05.618544 32749 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.638865 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.656324 32749 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.658025 32716 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:05.658473 32716 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.658876 32716 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.626433 32725 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.656656 32750 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 12:24:05.661171 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 166365us
W20260709 12:24:05.662824   312 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:24:05.668176 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:05.659648 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 180920us
I20260709 12:24:05.674628 30987 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:05.654614 32496 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:05.677170 32496 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.677841 32750 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:05.678253 32750 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:05.678472 32496 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.679435 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:05.679273 32750 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.680805 32660 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:05.680732 32559 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:05.681342 32660 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.681537 32750 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.681804 32660 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.681897 32559 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:24:05.682497 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 43919.3us
I20260709 12:24:05.631950   303 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.682413 32559 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:05.678108 32716 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.679319 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.641753   304 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.680821 32652 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:05.691267 32652 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.691653 32652 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:05.685593 32720 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.001s
I20260709 12:24:05.685026 31062 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:05.644382 32659 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:24:05.693199 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.694726 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.696784 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.687608 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1658) took 1463 ms. Trace:
I20260709 12:24:05.659720 32754 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:05.695631 31139 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:05.696965 31061 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.692081 32716 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.697613 30987 rpcz_store.cc:276] 0709 12:24:04.211398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.211613 (+   215us) service_pool.cc:224] Handling call
0709 12:24:05.604829 (+1393216us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:05.675169 (+ 70340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4256640}
I20260709 12:24:05.699394 32720 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.702476 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.703990 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.704527 32720 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.705412 32725 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.705854 30985 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:05.706619 32700 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.709542 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.709852 32731 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:24:05.711376 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.706133 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.716198 31064 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.716429 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:05.717409 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.700120 32754 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:05.719005 32754 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.719908   303 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.721925 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.723872 32763 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.724573 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:05.720889 32725 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:05.725390 31065 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.726013 31143 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:05.726267 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.726953 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.727402 32713 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.727952 32713 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.728446 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:05.727030 32731 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:05.727902 31144 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 3: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:24:05.715521 31139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1700) took 1642 ms (client timeout 2002 ms). Trace:
I20260709 12:24:05.669462 32740 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 12:24:05.733042 31139 rpcz_store.cc:269] 0709 12:24:04.058970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.243169 (+184199us) service_pool.cc:224] Handling call
0709 12:24:05.639991 (+1396822us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b580040c1d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb7f00aa 00007f2eeb872447 00007f2eeb88afe7 00007f2eeb88c7de 00007f2eeb89827c 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c
0709 12:24:05.701306 (+ 61315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2874128768}
I20260709 12:24:05.733814 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:05.701122 32729 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:05.720923 32667 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:24:05.726064 32725 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:05.749557 32763 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:05.751125 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.752336 31061 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:05.758299 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:05.764784 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:05.754422 32659 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.749081   303 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.775169   318 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:05.776428   317 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:05.780030   317 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.764065 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:05.752707 32725 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:05.778959 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 72928.8us
I20260709 12:24:05.771180 32713 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:05.778245 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 94728us
I20260709 12:24:05.770051 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:05.757786 32661 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.721009 32695 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.771687 32749 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.714973 32758 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:24:05.824653 31068 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:05.785200 32740 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:05.781247   317 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:05.830292 32729 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.832078 32725 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.833609 32725 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:05.780441 32559 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.783665 32659 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:05.837179 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 78758.3us
I20260709 12:24:05.788513   318 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:05.838421 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 51470.8us
I20260709 12:24:05.843312 32526 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:24:05.840922   304 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:05.828071 32661 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.848898 31106 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:05.851092 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 97904.8us
I20260709 12:24:05.788656 32727 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.844594 32721 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.858916 32763 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:05.858997 32559 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.859908 32763 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:05.862792 32655 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:05.863277 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:05.863852 31064 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:05.863989 31062 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:05.864640 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.865141 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:05.864513   318 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.870863 32630 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:05.871950 32630 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.872299 32630 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:05.873170 32630 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.873561 32720 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:05.874110 32720 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:05.875080 32630 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.876181 32704 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:05.876735 32713 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:05.876886 32661 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.877149 32713 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:05.877542 32713 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:05.878293 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.878064 32608 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:24:05.881070 32680 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:24:05.894618 32656 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:05.895066 32656 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:05.848593 32695 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:05.897042 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 51630.1us
I20260709 12:24:05.898806 31062 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:05.901271 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 52715.7us
I20260709 12:24:05.862526 32659 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.930349 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:05.932330 32713 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:05.855147 32675 consensus_queue.cc:1175] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce27a72886964e4287059a1d66bd5b6b" responder_term: 1 status { last_received { term: 1 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 1 index: 3 } }
I20260709 12:24:05.935673 32731 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.935745 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.936283 32731 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:05.937237 32731 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.938961 32713 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.939009 30889 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 194) took 2238 ms. Trace:
I20260709 12:24:05.941129 32652 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.875231 32720 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.899003 32741 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:05.939509 30889 rpcz_store.cc:276] 0709 12:24:03.700232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.700368 (+   136us) service_pool.cc:224] Handling call
0709 12:24:05.938804 (+2238436us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:05.387035 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:05.388137 (+  1102us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:05.388200 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:24:05.388233 (+    33us) tablet.cc:662] Decoding operations
0709 12:24:05.388824 (+   591us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:05.388932 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:05.388985 (+    53us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:05.389356 (+   371us) tablet.cc:701] Row locks acquired
0709 12:24:05.389406 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:24:05.389766 (+   360us) write_op.cc:270] Start()
0709 12:24:05.389961 (+   195us) write_op.cc:276] Timestamp: P: 1783599845389614 usec, L: 0
0709 12:24:05.390007 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:24:05.391452 (+  1445us) log.cc:844] Serialized 595 byte log entry
0709 12:24:05.770627 (+379175us) op_driver.cc:464] REPLICATION: finished
0709 12:24:05.841868 (+ 71241us) write_op.cc:301] APPLY: starting
0709 12:24:05.841963 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:05.842071 (+   108us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:05.843342 (+  1271us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:05.843641 (+   299us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:05.843923 (+   282us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:05.843981 (+    58us) write_op.cc:312] APPLY: finished
0709 12:24:05.850756 (+  6775us) log.cc:844] Serialized 23 byte log entry
0709 12:24:05.850939 (+   183us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:05.851016 (+    77us) write_op.cc:454] Released schema lock
0709 12:24:05.851073 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:05.851490 (+   417us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:05.851516 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":292763,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":68879,"apply.run_cpu_time_us":2784,"apply.run_wall_time_us":9621,"num_ops":1,"prepare.queue_time_us":172744,"prepare.run_cpu_time_us":13889,"prepare.run_wall_time_us":30473,"raft.queue_time_us":80454,"raft.run_cpu_time_us":6818,"raft.run_wall_time_us":17325,"replication_time_us":380441,"thread_start_us":249927,"threads_started":4,"wal-append.queue_time_us":103025}]]}
I20260709 12:24:05.942839 32720 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.896031 32656 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.947129 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:05.942477 32496 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:05.950798 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.951169 32741 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:05.952664 32680 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:05.953169 32680 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:05.953349 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.954499 32680 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.956342 32680 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.960794 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:05.962220 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:05.946667 32731 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.966811 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:05.965054 32721 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 12:24:05.970569 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 65191.5us
I20260709 12:24:05.973405 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.976138 31058 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:05.907503 32754 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:05.985824 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:05.990312 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.988754 31106 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:05.972476 32608 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:24:06.011799 32648 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.011723 31696 raft_consensus.cc:2955] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:06.016533 30984 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:05.983530 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:05.944305   317 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.029510 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:06.011282 32713 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:06.048255 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:06.049135 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:06.052325 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.995878 32660 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.008409   318 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.058825 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:06.030879 32692 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.061590 32656 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.058234 32729 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.076130 30989 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:06.088543 31060 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:06.091063 32646 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:06.095515 32645 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:24:06.096841 32704 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:06.100314 30990 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.100579 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:06.087759 32714 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.113322 30988 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:06.117336 30989 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.120146 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.121170 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.117242 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:06.125114 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.125372 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:06.125061 32714 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.127888 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:06.129540 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1619) took 1124 ms. Trace:
I20260709 12:24:06.130396 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:06.131942 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:06.132164 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:06.129930 31060 rpcz_store.cc:276] 0709 12:24:05.005093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.009494 (+  4401us) service_pool.cc:224] Handling call
0709 12:24:05.009747 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.090832 (+1081085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.129289 (+ 38457us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b5400383454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:06.129488 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2230610304}
I20260709 12:24:06.150525 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.151574 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:06.120239 32729 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:06.155218 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.155148 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:06.155534 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:06.157509 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:06.158128 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:06.147405 32648 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:24:06.159297 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:06.148900 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:06.162624 32730 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:06.166786 31139 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:06.155375 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:06.167388 32754 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:06.137683 32652 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:06.172326 30986 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
W20260709 12:24:06.188480 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 102330us
I20260709 12:24:06.188768 31062 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:06.194375 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:06.194957 31064 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:06.193991   325 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:24:06.199047 30876 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet d50d1465e4b84136bf90de700a93ef2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
W20260709 12:24:06.206842   323 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:24:06.207583 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:06.182998 32730 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:06.199774   303 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.209898 32730 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.205489 30990 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:06.211373 32730 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.214250 31059 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:06.215840 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.217152 32762 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.218091 32676 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:06.220269 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.221419 32659 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:06.221608 30958 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.222309 32659 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:06.223531 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:06.226257 32762 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.226302 32740 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:06.226835 32740 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:06.227685 32740 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.228969 32740 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.230988 32675 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:06.231359 32675 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:06.232226 32675 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.233513 32675 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.235113 32660 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:06.172374 32729 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.237608 32729 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:06.238500 31068 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:06.244360   317 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:06.247223 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.248879 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.251204 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.251296 30958 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:06.258622 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.260799 32727 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.261809 32688 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:06.262210 32688 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:06.262369 32655 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:06.263209 32688 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.264467 32688 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.264662 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.266507 32743 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:06.266870 32743 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:06.267679 32743 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.268616 32743 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.254680   321 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:06.254891 32608 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:06.273331 32608 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.275317 32608 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:24:06.270131 32648 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:24:06.284250 30885 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 310) took 1941 ms. Trace:
I20260709 12:24:06.284621 30885 rpcz_store.cc:276] 0709 12:24:04.297326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.297512 (+   186us) service_pool.cc:224] Handling call
0709 12:24:06.238155 (+1940643us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":918996,"reported_tablets":2}
I20260709 12:24:06.285504   328 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:06.290249   335 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:24:06.253211 32660 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.303248 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.303552 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:06.311378 31067 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:06.311769 30986 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:06.315672   328 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:06.316732   328 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:06.317051   328 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:06.319002 30986 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.322744 31064 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
W20260709 12:24:06.326328 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68570.7us
I20260709 12:24:06.327330 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:06.329202 32714 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:06.332864 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:06.333444 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.256451 32695 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:24:06.337370 31062 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:24:06.324153   303 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.342926 31068 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:06.346931   303 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.348961   303 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:06.334396 32688 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:06.359748 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:06.349826 32716 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:06.317844   328 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.360497 30988 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:06.330444   318 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:24:06.349709 32714 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:06.354794   321 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:06.373449 32700 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:06.374050 32526 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:06.358211 32688 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:06.380946 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:06.362866 32716 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:06.381744 30984 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:06.387365 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:06.525274 32716 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:06.529016 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:06.529825 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.390689   328 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:06.541558 32744 consensus_queue.cc:1175] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce27a72886964e4287059a1d66bd5b6b" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
W20260709 12:24:06.539086 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 158471us
W20260709 12:24:06.551986 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 154327us
I20260709 12:24:06.380154   303 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.162s
I20260709 12:24:06.384724 32714 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.591519   303 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:06.591926   303 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:06.592839   303 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.563277 32741 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:06.617110   331 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:24:06.617782 30992 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.538298 32648 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 12:24:06.624598 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 60688.1us
W20260709 12:24:06.584316   310 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.061s	user 0.000s	sys 0.003s
I20260709 12:24:06.551963 32661 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:06.632913 32721 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.594092   303 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.644034 32648 consensus_queue.cc:1243] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:06.709983 32661 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.713016 32661 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:06.714700 32714 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:06.716605 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.717394 31138 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:06.718815 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:06.720604 31695 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:06.584988 32688 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.688776 32721 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.730343 32714 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:06.743897 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 52196.8us
I20260709 12:24:06.750511 32606 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:06.750792 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:06.751039 32606 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:06.752010 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:06.752300 32606 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.753892 32606 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.754395 32741 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:06.755067 32741 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:06.755628 32741 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.756039 32741 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.757834 32741 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:06.753860 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:06.740141 32704 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:06.764703 32704 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.766618 32704 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:06.769761 32721 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:06.770972 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.771194 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:06.771698 31142 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:06.771804 31059 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
W20260709 12:24:06.772569 32675 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.002s
I20260709 12:24:06.772899 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:06.774725 32727 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.000s
I20260709 12:24:06.746078 32714 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:06.778731 32714 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.781190 32721 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.781580 32721 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.781910 32721 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.785118 30987 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:06.790321 32661 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:06.804446 32730 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.805124 32730 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.808732 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:06.810582 32730 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:06.812124 31059 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:06.813687 31068 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:06.820390 32714 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.822507 32714 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:06.832221 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.835304 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:06.836660 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 82863.5us
I20260709 12:24:06.838764 31142 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:06.839779 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:24:06.842430 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.847404 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:06.848193 31690 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 2: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:06.854180 32652 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:06.854574 32652 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:06.846779 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:06.863631 30889 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 4, VOTER 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) evicted. New cstate: current_term: 1 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:06.834554 32727 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:24:06.840373 32743 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.872954   321 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.837559 32714 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:24:06.881096 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:06.882181 31135 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:06.883040 32704 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.883483 32704 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.885434 32704 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:06.887070 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.888082 32675 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:06.888520 32675 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:06.889317 32675 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.891395 31139 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:06.891844 31139 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:06.892470 31692 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 2: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:06.892799 31139 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:06.894304   348 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:24:06.899335 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:06.894022 31139 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:06.907783 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 69763.7us
I20260709 12:24:06.873420   321 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.871066 32743 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.917162 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.875043 32727 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:06.907235 32716 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:06.924875 32727 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.926991 32727 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:06.928311 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:06.930116 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:06.934513 31067 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:06.915647 32680 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.916970 32743 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.948875 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:06.970624 31139 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:06.973387 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:06.974109 30989 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:06.974489 30985 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.948751 32762 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:06.952500 32680 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.949523 32727 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:06.979574   356 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:06.980487   355 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:06.962556 32744 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.983997 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:06.971151   318 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:06.977636 32525 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:24:06.952843 32714 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:24:06.986240 32645 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:06.988512 32762 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:06.989216   321 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:06.988940 32744 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.043560 32645 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.028398 32727 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.027810 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.167310 31139 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:24:06.983362 32680 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.165320 32762 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.181116 32652 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:06.981765 32716 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:07.028038   356 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:06.990543 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:07.161604   328 consensus_meta.cc:287] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.001s
I20260709 12:24:07.161435 32721 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.161901 32654 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:07.196346 32654 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.198328 32654 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.198372 32645 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.200419 30985 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:07.200260 32716 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.205845   356 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.211624 32721 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.162824 32744 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.162940   301 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:07.032465   304 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 12:24:07.161384 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 127821us
I20260709 12:24:07.185674 32688 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:07.186913 31064 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:07.189997 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1671) took 1163 ms. Trace:
I20260709 12:24:07.219079 30987 rpcz_store.cc:276] 0709 12:24:06.026474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.026635 (+   161us) service_pool.cc:224] Handling call
0709 12:24:07.189819 (+1163184us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b5400226d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:07.189928 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646084352}
W20260709 12:24:07.214589 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 333545us
W20260709 12:24:07.190764 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.193053 32652 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:07.194331 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 201513us
I20260709 12:24:07.176448 32727 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.222824 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.232575 31139 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:07.223968 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.233582 30985 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:07.221967 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.229184 31067 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:07.231931 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.230898 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:07.237450 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.238276 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:07.238819 31064 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.240506 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:07.237387 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:07.235127 30991 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:07.238503 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.252357 30986 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:07.255357 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:07.257175 31696 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:07.257743 31135 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:24:07.270082 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.251761 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.242308   350 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:07.259776 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.226362   301 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.252823 32744 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.273993 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:07.329380 31138 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.329031 32744 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.329663   301 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.306658   322 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:24:07.306434   361 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.331429 32744 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.331552 31691 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:07.258100 32692 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:07.313949 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.257517   360 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.268433 32675 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.258036 32727 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.334566 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.277855 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.342118 31141 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:07.284940 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:07.276090 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:07.344024 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59193.7us
I20260709 12:24:07.338195 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.345628 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.353895 31064 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:07.309988 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.358003 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:07.332252 32692 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.273572   359 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:07.275712   304 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:07.340914 32675 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.409569   318 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.409204 32721 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:07.386775   362 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.382151   361 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.433470 32675 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.388849   359 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:07.436749   359 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:07.437069   361 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.390779 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.440263 31135 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
W20260709 12:24:07.425315 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 155675us
I20260709 12:24:07.434072 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:07.313308 32680 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.407691 32716 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.414499   328 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.388667 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:07.407503   322 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.430361 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:07.348461 32744 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.382973 32655 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:07.385905   321 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.386883 32654 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:07.415576 32692 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.415764 32676 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:07.440304 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:07.392038 32656 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.406251 32661 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.442858 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:07.390604   363 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:07.346378 32727 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.348465   350 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.408543 31690 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:07.275473 32741 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:07.486507 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.457721 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.527174 31696 raft_consensus.cc:1813] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Handling vote request from an unknown peer 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:07.456823 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.486992 31697 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:07.514840 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 80855.6us
I20260709 12:24:07.356068   360 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.449189 32730 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:07.516009 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 75780.2us
I20260709 12:24:07.552771 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:07.457113 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:07.447093 31691 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:07.524426 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.448770 32744 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.526010 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.447765   304 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:07.442206 32721 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:07.497735 32743 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.526s	user 0.007s	sys 0.003s
I20260709 12:24:07.445909 32716 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.498178 31068 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:07.527004 32704 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:07.554755 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 166153us
I20260709 12:24:07.523558 32692 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:24:07.497089 32656 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.564129 30991 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:07.580865 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.498533   303 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:07.443940   318 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.520359   356 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:07.515271 32608 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:07.566524 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.551623 31696 raft_consensus.cc:2410] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:24:07.564214   320 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.574611 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.526835 32741 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:07.571593 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.563962 32525 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:07.583366 31063 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:07.567252 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.551990 32648 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:07.448567   328 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.514961   321 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.566574 32744 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.449529   322 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.498323 32655 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.594000 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:07.595068 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.608997 30984 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:07.568645   355 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.568951   371 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:07.615367 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 158338us
I20260709 12:24:07.448082   362 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.568919 32676 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:07.524935 32654 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.596140 32741 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.526330 32661 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.605782 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.584008 32656 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.584404   303 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:07.589569   320 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:07.630900   320 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:07.631048   366 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:24:07.576678 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:07.635136   303 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.637041 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.637815 31063 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:07.584969   304 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.572113 32743 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.648838 30986 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:07.629551 32714 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:07.614470 32648 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.609943 32525 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:07.630402   360 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.639087 31135 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:07.624987 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.640117 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:07.610110   317 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:07.530959   363 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.615042   355 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.626919   361 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.627347   375 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.624207 32676 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:07.678200 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.601575 32608 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.668536 32721 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.669065   304 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.669556 32692 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:07.671653 32648 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.672451 32525 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.674641 32654 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.707067 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:07.727687 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:07.686977   364 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.268s	user 0.001s	sys 0.001s
I20260709 12:24:07.733659 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.736573 31060 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:07.733247 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.662158 32762 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.738237 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.747118 31058 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:07.743505 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.748226 31695 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:07.685135 32645 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.696269   373 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.754535 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:07.688843 32758 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.626526   371 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.684494   303 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.750591 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.752527 31065 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:07.692876 32661 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.685389 32660 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:07.756644 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.766176 31058 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:07.687983 32655 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:07.758536 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.768903 31060 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:07.759644 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.688591 32727 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.694908   321 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.681740 32716 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.691305   322 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.684067   362 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.699925 32675 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.688576   355 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.693921 32676 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.754184 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.775015 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:07.759816   373 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.780038   360 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.779910 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.686753   363 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.685310   317 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.691287   361 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.768324 31696 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:07.699887   320 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.760845 32680 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:07.693804   359 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.754789 32721 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:07.798143 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.787919 30987 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:07.784657 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.788718 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.811501 31697 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:07.811756 31063 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:07.765084 32660 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.691663   375 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.791579   377 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:07.792506 31691 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:07.794202 31692 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 12:24:07.814708 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 220806us
I20260709 12:24:07.802117 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.789860   373 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.783423   382 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.826406 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:07.765789 32762 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.764108 32758 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.785247 32645 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.839573   361 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.829684 32744 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.843277 32554 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.849713 32554 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.850551 32744 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.851140 32744 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.851583 32744 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:07.852700   370 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
I20260709 12:24:07.853595 32744 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.853788 32554 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:07.791006   360 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:07.855451 31058 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:07.847958   328 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:07.845269 32652 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.614s	user 0.008s	sys 0.000s
I20260709 12:24:07.839406   317 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.824047   350 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:07.908792   386 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:07.953070   386 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.955868   375 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:07.956970   372 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
W20260709 12:24:07.958722   378 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.278s	user 0.000s	sys 0.002s
I20260709 12:24:07.797938   380 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:07.963521   380 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.964810 31062 raft_consensus.cc:1217] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:24:07.965312 31062 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:24:07.824685 32660 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:07.966701   320 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.967983   359 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:24:07.912093 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:07.970278 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:07.970912 31059 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.971333 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.856788 32730 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:07.864130   361 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:07.920581 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 166121us
I20260709 12:24:07.974519   361 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.948395   384 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:07.972085 31064 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:07.824950 31138 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:07.839808   371 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.927119 31139 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.828291   382 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:07.846328 32675 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.864454   360 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.951468 32743 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.827889   377 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.930318 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.863332   304 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:07.828804   318 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.936940 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.938011 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.940387 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.982551   384 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.922596 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.940595 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.847747 32656 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.940387 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.942600 30955 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:07.942804 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.943109 32645 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:07.981698   382 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.944134 31106 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:07.944762 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.946857 31689 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:07.986609 31142 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:07.947407 32676 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:07.947659 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 199444us
I20260709 12:24:07.858878 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:07.826843 32721 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.961773 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:07.972326 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.980099 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.973451 32730 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.053365   355 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.940546 32704 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.057214   355 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.959985   301 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:08.058986   301 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.057564 31144 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:08.059327   355 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.024933 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.060985   301 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.989512   384 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.061322 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:07.987458 31137 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:07.987708 31058 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:07.995141 32714 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:07.995472 31141 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:08.063452 32714 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.063843 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.064555 31058 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.063856 31144 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:08.065685 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.065900 32743 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.066370 32743 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.066823 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.067538 30992 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.067682 32721 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.068379 32721 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:08.025413 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.055830   386 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.069031 32721 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.069384 31695 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.069844 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.984174 32608 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:07.982788   392 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.070453 31064 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:08.070899 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.070874   392 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.072156 31695 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.069424 31142 raft_consensus.cc:2368] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.073602   392 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.062011 32675 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.074421 32675 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.985364 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.052802 32676 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:07.980001   380 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.071311 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.012869 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.075960 31064 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.076637 32676 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.979876   359 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:08.078595 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.078873   359 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.074995 32675 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.079456 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.079859 31698 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.081103   359 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:07.995142 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 51094.9us
I20260709 12:24:08.053124 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.083303 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.083650 32652 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.083920 30984 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:08.084041 32652 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:08.084412 32652 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.988329 31691 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:08.085935 32762 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:08.062557 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 120029us
I20260709 12:24:08.086840 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.088104 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:07.990255 31143 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:08.089589   360 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:07.988577 31135 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:07.989935 32656 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:07.993045 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.091084 32762 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.091234 32652 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:08.091630 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 232810us
I20260709 12:24:08.119810 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:07.990387 31690 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:24:08.092186 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:07.984287 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.093258 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.068413 32743 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.034108 31139 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:08.094491 31696 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:08.094856 31061 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.094556 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.070927   318 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.104223   359 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.095679 32727 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.987159 31068 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:08.052591 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.102558 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.137554 31058 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:08.138662 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:07.990558 31060 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:07.983943 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.096431 31065 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:08.099794 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.096232   393 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.111629   320 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.997265 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:07.987807   377 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.142032 30991 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.061559 31059 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.143910 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.103040 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.144593 31063 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.111750 32676 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:07.986472   381 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.300s	user 0.002s	sys 0.000s
I20260709 12:24:08.078611 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:07.983686 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.092572 32741 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.126847 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.165988 30990 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:08.127449 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.127514 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:08.168363 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:08.168803 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.065461 32714 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:07.983632   361 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.141636 32656 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.176791 32656 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.130316 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.131366 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.141172 31661 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.178887 32656 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.179303 30984 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:08.141772 32727 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.104842 30985 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:08.143569   386 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.024190 32645 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.141772   304 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.123036 31138 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.164239 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.132969   318 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.132851   380 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:08.173897 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.174289 32721 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.175354 32762 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.199172 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.199451 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.199754 31135 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.180284 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.136504 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.183094 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.146157   320 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.146751   393 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.147004   355 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.172626 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:08.180794   303 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.188757 30992 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.190655 31139 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:08.184159 32727 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.204389 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.180078 31689 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:24:08.185876 31693 raft_consensus.cc:2368] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.214488 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.215790 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.216073 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.216423 31137 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:08.216648 31142 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:08.216758 31143 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:08.217434 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.217590 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.217754 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.195174 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.169984 32654 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.196693 31065 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:08.200758 31062 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.200752 30985 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:08.147550   321 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.238610   321 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:08.239032   321 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.240933 32645 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.183401   384 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:24:08.242482 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.195806   380 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:08.243422   380 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.234834 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.244742 30958 leader_election.cc:400] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.245064 30958 leader_election.cc:403] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.245718 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.195613   318 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.246979   379 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.247622   379 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.247923   379 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.248232   379 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.183251   386 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.249670 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.250005   386 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.145453   359 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.250523 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.251102 30985 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:08.252146   359 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.251948   382 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.252789 31697 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:08.254460 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.255061 31692 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:08.255923 32730 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.256358 32730 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.256681 32730 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.204960 31696 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:08.259215 32675 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.194545 31067 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.205515 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:08.260280 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54835.4us
I20260709 12:24:08.260886 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.261229 32675 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.185542   377 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.261430 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.207074 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.208609 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:08.209054 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 92064.7us
I20260709 12:24:08.251983 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.209632   376 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:08.214569 32656 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.233721 32660 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.285705 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.216357 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.230866 30992 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:08.301776 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.302489 31696 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.232578 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.303164 30988 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:08.202935   393 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:08.233011 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 117271us
I20260709 12:24:08.218103 31144 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:08.207437   304 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.254009   382 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.163738 32676 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.237517 31135 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.262902 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.268163   359 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.274809 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:08.195336 31061 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.279469 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:08.274204 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.203080   355 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.202646   362 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.218972 31142 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:08.282517 30991 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:08.317263 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.317848 31064 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:08.231848 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.305790 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:08.218060 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.234129 32741 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.321098 30990 raft_consensus.cc:2368] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:08.235167 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.219782 32652 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.300644   384 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:08.314035 31062 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:08.313912 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.274258 30985 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 2: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:08.248128 32758 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.206382   320 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.309460 32608 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:08.354173 31060 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.318763 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.323678 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:08.322641 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 104660us
W20260709 12:24:08.287006 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 79998.5us
I20260709 12:24:08.281584   377 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.327003 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.312638 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.327787 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.358565 31141 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
W20260709 12:24:08.328341 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48915.8us
I20260709 12:24:08.203585   373 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.333658 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.282332 31143 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
W20260709 12:24:08.366451   400 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:24:08.359292   377 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.367313   391 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:08.334975 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 60858us
I20260709 12:24:08.309298   304 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.369148   304 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.359805   382 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:08.327659   362 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.354008 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.374356 32716 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:08.339008 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.262606 32743 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.342653 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.379375 31065 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:08.343329 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.343755 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.313330 32645 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:24:08.344542   318 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.345448 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.338335   317 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.338060 31696 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:08.347270 32660 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.304551 32656 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.350564 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.335155 32714 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.357044 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.358799 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.361164 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.397513 31068 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.397445 31033 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.361917 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:08.318066   355 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.362865 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.396440 30955 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.400641 30983 raft_consensus.cc:2368] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.398861 30984 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:08.365306 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.365803 32608 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.401430   303 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.407326   303 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:08.407685   303 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.408985 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.409916 31689 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:24:08.410640   401 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:08.416733 32660 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.417161 32660 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.417495 32660 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.375944 31138 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.422389   350 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:08.422968   350 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:08.424022   350 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.374497   362 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.341887 32676 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.425532   350 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.269634   386 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.358187 31142 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:08.320974   375 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:08.439926 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 74684us
I20260709 12:24:08.438548 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.440698 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:08.378180 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.389328 31695 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.394256 30990 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:08.374840 31137 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.395094 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:08.387405   317 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.361482 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.401311   392 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.453831   392 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:08.395792 32714 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.454406 32714 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.318969   359 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:08.462393   359 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.465299   392 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.456405 32676 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.445428 32680 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.447889 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.451995   384 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.468765   384 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.468918 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.469161   384 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.470376 32744 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.470857 32744 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.471218   391 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.471594   391 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.471971   391 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.471726 32744 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.387917   360 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.475853 32554 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.476282 32554 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.379209 32743 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.482429 32743 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.483134 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.452987 32741 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:08.457113 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 62092.9us
I20260709 12:24:08.460153 31698 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:08.486917   318 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.489166   386 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.491330   373 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.492735 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.460104 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:08.493475 32743 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.494208 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:08.495877 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.495992 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.496443 31065 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.456284 32654 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.452903   317 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.498962   377 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.441116 32652 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.500072 32652 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:08.500500 32652 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.500999   377 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.495293 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:08.502417 31664 leader_election.cc:400] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.503281 31664 leader_election.cc:403] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.503937 32680 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:08.503859   304 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.504513 32680 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.505085 31062 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.505443 32680 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.505942   304 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.476694 32721 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:08.521356 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 64221.8us
I20260709 12:24:08.521840 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.522607 31136 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.480762 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.481868 32656 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.502974 31691 raft_consensus.cc:2368] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
I20260709 12:24:08.477717 32554 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.506477   375 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.503361   380 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.508389 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.531502 31062 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.478461 32608 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.509749   328 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.511703   355 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.512408 32714 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.480770 32730 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.544873 31136 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:08.545709 32730 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.512938   379 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.513624   373 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.547801 31140 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:08.516753   384 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.518119 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.518445   321 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.518749 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.550297 31690 raft_consensus.cc:2368] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.551273   373 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.564529 32554 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.565236   373 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.566686 32554 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.555568 30889 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 196) took 1708 ms. Trace:
I20260709 12:24:08.519958   320 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:08.523036 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 126659us
W20260709 12:24:08.524044 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 162193us
I20260709 12:24:08.524286 31109 leader_election.cc:400] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:08.528004 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.581565 31109 leader_election.cc:403] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.581744 31142 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.582011 30958 leader_election.cc:400] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
W20260709 12:24:08.582280 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58124.9us
I20260709 12:24:08.529203 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:08.583528 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 54397.6us
I20260709 12:24:08.585582 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.527705 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:08.535440 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:08.520630 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.531162 32704 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.588531 32704 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.588574 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.589027 32704 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.589344 31061 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.590154 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.590683 30990 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.537541   375 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.591732   375 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.592662   350 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:08.593487   406 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:08.539243 30986 raft_consensus.cc:2368] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.595831 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:08.597405 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.597951 31140 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.598600   334 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.598987   334 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:08.599344   334 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.540777   355 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.603513 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:08.522429 32721 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:08.537482   388 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:24:08.542826   303 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.543934 31062 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.551290 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.611747 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.550801 30983 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.545403   359 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.552345   393 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.553224 32714 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.519328 31065 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.553619   317 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.555526   380 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.555658 31049 tablet_service.cc:1558] Processing DeleteTablet for tablet d50d1465e4b84136bf90de700a93ef2c with delete_type TABLET_DATA_TOMBSTONED (TS 2c4511d66b474ee0aa41c9b1f03b66ab not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51930
I20260709 12:24:08.638139 31060 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.550884   391 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.557497 32743 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.558316 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.558867 32652 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.640353 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.641448 32652 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.530213   386 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.642170   386 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.644348   386 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.531908 32727 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.645356 30988 raft_consensus.cc:2368] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
I20260709 12:24:08.561522   328 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.532171 32660 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.618736   301 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:08.647467   317 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.648571 32660 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.563900   379 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.569911 30889 rpcz_store.cc:276] 0709 12:24:06.846882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.848940 (+  2058us) service_pool.cc:224] Handling call
0709 12:24:08.555492 (+1706552us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:07.348374 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:07.498696 (+150322us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:07.515033 (+ 16337us) write_op.cc:435] Acquired schema lock
0709 12:24:07.515062 (+    29us) tablet.cc:662] Decoding operations
0709 12:24:07.581325 (+ 66263us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:07.585605 (+  4280us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:07.588621 (+  3016us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:07.616152 (+ 27531us) tablet.cc:701] Row locks acquired
0709 12:24:07.619684 (+  3532us) write_op.cc:260] PREPARE: finished
0709 12:24:07.638212 (+ 18528us) write_op.cc:270] Start()
0709 12:24:07.653582 (+ 15370us) write_op.cc:276] Timestamp: P: 1783599847627016 usec, L: 0
0709 12:24:07.666710 (+ 13128us) op_driver.cc:348] REPLICATION: starting
0709 12:24:07.689377 (+ 22667us) log.cc:844] Serialized 533 byte log entry
0709 12:24:07.924452 (+235075us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2ee878a0ac 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b
0709 12:24:08.409299 (+484847us) op_driver.cc:464] REPLICATION: finished
0709 12:24:08.548085 (+138786us) write_op.cc:301] APPLY: starting
0709 12:24:08.548163 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:08.548270 (+   107us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:08.548300 (+    30us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:08.548611 (+   311us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:08.548763 (+   152us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:08.548810 (+    47us) write_op.cc:312] APPLY: finished
0709 12:24:08.548985 (+   175us) log.cc:844] Serialized 23 byte log entry
0709 12:24:08.549137 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:08.549227 (+    90us) write_op.cc:454] Released schema lock
0709 12:24:08.549261 (+    34us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:08.549585 (+   324us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:08.549611 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10348,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":138560,"apply.run_cpu_time_us":1473,"apply.run_wall_time_us":1474,"num_ops":1,"prepare.queue_time_us":405997,"prepare.run_cpu_time_us":64312,"prepare.run_wall_time_us":781202,"raft.queue_time_us":176665,"raft.run_cpu_time_us":10812,"raft.run_wall_time_us":122566,"replication_time_us":742439,"spinlock_wait_cycles":84558080,"thread_start_us":750778,"threads_started":4,"wal-append.queue_time_us":476874}]]}
I20260709 12:24:08.587038 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:08.587245 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 51884.1us
I20260709 12:24:08.587911 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.614665   392 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:08.650302 31063 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:08.623241 31059 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.601521 31664 leader_election.cc:400] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.604444 31142 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.605007 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:08.663893 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 62498.5us
I20260709 12:24:08.664191 31064 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.664597 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.665652 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:08.666389 32608 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.666837 32608 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.667716 32608 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.669356 32744 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.602419 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.670467 31061 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.601418 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.671365 31068 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
W20260709 12:24:08.608088 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 80484.2us
I20260709 12:24:08.608624   382 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.552150   321 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.612581 31661 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.616367 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:08.623492 30987 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:08.578514 30885 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 4, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:08.628193 31137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:08.630059 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.630604 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.631770 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.678510 31062 raft_consensus.cc:2368] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.614094 32741 raft_consensus.cc:2764] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:24:08.684368 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.684828 31135 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.600729 32730 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:08.636138 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.686861 31065 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.689777 31141 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:08.631117 32676 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:08.690424 32676 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:08.691313 32676 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.692523 32676 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.693598 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.694523 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.694908 31143 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:08.695300 31692 raft_consensus.cc:2368] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.695935 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.696136 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.636598 31140 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.696480 31144 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.696173 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.697371 31693 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.610561 32721 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.701014 31065 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.639354 31067 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.639544 30985 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:08.650426 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.651634 31058 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.600608 30990 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.657030 32704 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.659227   303 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.660338   320 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.660733   391 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.711633 32721 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.713620 32721 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.713850 31135 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.649638 32727 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.662490   384 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.674520 30983 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
W20260709 12:24:08.675740 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 63235.8us
I20260709 12:24:08.726590 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:08.676146 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 59892.5us
I20260709 12:24:08.675302   382 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.733616   382 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.733798 32744 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.734339 32744 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.735176 32744 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.676636 30885 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 4, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 4 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:08.677423 31063 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:08.738739 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.738981 31060 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:08.738593 30885 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:08.739264 31063 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.739987 30885 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 4, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 4 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.679350 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.740366 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.743022 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.743703 31068 raft_consensus.cc:2368] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3.
I20260709 12:24:08.743641 32704 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.744576 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.745667 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.677085 32743 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.682284 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.746524 32743 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.747378 32743 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.741415 30885 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.750641 31106 leader_election.cc:400] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3.
I20260709 12:24:08.750931 31106 leader_election.cc:403] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.751017 30885 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:08.727459 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:08.753494 31698 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:08.682495 31064 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
W20260709 12:24:08.702376 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 52044.6us
I20260709 12:24:08.754637 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.755368 31063 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.755679 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.705906 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.701925 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:08.701315 31137 raft_consensus.cc:606] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:08.659231   393 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.759269   384 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.759704   384 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:08.760075   384 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.677599 31690 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.762342 32727 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:08.762897 32727 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:08.763557 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.763748 32727 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.764989 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:08.681562 32660 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.766398 32660 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.767458 31143 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:08.714056 31067 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.768442 32660 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.716087 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.718748 31144 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.769205 31062 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:08.769575 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.771898 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:08.773139 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.773424 31062 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:08.723541   328 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:24:08.776137   388 raft_consensus.cc:2764] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:08.720023   359 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:08.781136 31137 raft_consensus.cc:614] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:08.721261   334 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.783519 32762 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:08.784008 32762 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.784293   334 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.677138   304 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:08.785035   304 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:08.786981 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:08.786199   304 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.788373   304 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.785912 32762 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.789489   303 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:08.683943 32655 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.790092   303 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:08.701465 31138 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:08.790755   303 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:08.714850 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.791504 31058 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.722940 31695 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.793996   376 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:08.797186 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:08.798108   328 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.798823   328 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.800726   328 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:08.804291 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 102436us
I20260709 12:24:08.805650 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.807566 30983 raft_consensus.cc:2368] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3.
W20260709 12:24:08.811081 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 83685.2us
I20260709 12:24:08.811923 31698 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:08.731801 31061 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:08.755827 31068 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:08.687501 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:08.756448 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 50594.5us
I20260709 12:24:08.829857 30958 leader_election.cc:400] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.830256 30958 leader_election.cc:403] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.706564 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.830854 31139 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.830940 31062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49556
I20260709 12:24:08.756702 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.757184 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:08.831511 31062 raft_consensus.cc:606] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.756642 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.831725 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.832545 31063 raft_consensus.cc:2368] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.764295 31691 raft_consensus.cc:2368] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5.
I20260709 12:24:08.768594 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.661608   350 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:08.812098 31109 leader_election.cc:400] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
I20260709 12:24:08.840354 31109 leader_election.cc:403] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.718070 31059 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.813282 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.848901 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.850025 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.821331   382 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.822285   380 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.822196 32704 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3.
I20260709 12:24:08.855050 30989 raft_consensus.cc:2368] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3.
W20260709 12:24:08.822954 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 51939.7us
I20260709 12:24:08.823379   303 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:08.825766 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:08.831998 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 74899.9us
I20260709 12:24:08.830009 31140 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:08.833304 31664 leader_election.cc:400] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3.
I20260709 12:24:08.870460 31664 leader_election.cc:403] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.871166 31661 leader_election.cc:400] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.871464 31661 leader_election.cc:403] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.871552 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.870042 30983 raft_consensus.cc:2368] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.878715 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.879383 31068 raft_consensus.cc:2368] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
I20260709 12:24:08.879637   384 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.880441 30986 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:08.882195 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:08.882854 30992 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:08.887894 31138 raft_consensus.cc:2368] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3.
I20260709 12:24:08.888952   391 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:08.843766   320 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.843641   359 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.890952   359 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.888988   380 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:08.891875   359 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.892536   380 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.843379   373 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.898433   373 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:08.899271   373 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.831846 31062 raft_consensus.cc:614] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:08.869102 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:08.903633 31664 leader_election.cc:400] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.905463 31664 leader_election.cc:400] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3.
I20260709 12:24:08.908429 31136 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:08.834322 31060 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.912273 31139 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:08.752740 30885 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
W20260709 12:24:08.841115 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 153695us
I20260709 12:24:08.842272 30958 leader_election.cc:400] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.918830 30958 leader_election.cc:403] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 12:24:08.919427 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 77276.8us
I20260709 12:24:08.920547 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.921579 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.923494 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:08.924286 31063 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:08.728710 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:08.850991 30991 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:08.845168 31143 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.869968 31109 leader_election.cc:400] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
W20260709 12:24:08.930613 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 60783.1us
I20260709 12:24:08.931973 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.892197   384 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.845026   393 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.843571   334 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
I20260709 12:24:08.938371   384 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.839686   376 raft_consensus.cc:2764] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:08.938962   384 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:08.895609 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.897387   303 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:08.843987 32660 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 12:24:08.941227 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 101379us
I20260709 12:24:08.943275 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:08.943833 31694 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.945379 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.945832 30985 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:08.946985 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.951145 30983 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:08.902870 31693 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:08.838083 31695 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.842836 31058 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:08.942425 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.927065 31106 leader_election.cc:400] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
I20260709 12:24:08.952692 31106 leader_election.cc:403] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.927402 30955 leader_election.cc:400] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.930352 30885 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:08.953683 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:08.953984 30885 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:08.936851 31060 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:08.938251   393 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.957188 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.962303 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:08.962852 30991 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.960878 30885 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 2 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.843796   388 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 12:24:08.977200 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.930270 32654 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:08.933984   321 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4.
I20260709 12:24:08.977730 31140 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.979010 31109 leader_election.cc:400] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3.
W20260709 12:24:08.979431 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52449.2us
I20260709 12:24:08.937750   356 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:08.867625 32762 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 12:24:08.992944 30991 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:08.976630 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.995060 31058 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:08.941082   384 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:08.997987 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.998611 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:08.969314   393 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:09.001536 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.958441   391 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3.
I20260709 12:24:09.003085 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.959847   362 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.999578 31139 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:09.011821 31140 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:09.025113 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:09.008225 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:09.122992 31136 raft_consensus.cc:2368] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3.
I20260709 12:24:09.115937 31058 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.125180 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:09.000981   413 tablet_replica.cc:333] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
W20260709 12:24:09.097188 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 68885us
I20260709 12:24:09.153337 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:09.027204 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 108620us
I20260709 12:24:08.967453   386 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5.
I20260709 12:24:08.980937 32654 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.995882   303 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.998127 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:09.015810 31064 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:09.006268 31692 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:09.160086 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:09.172439 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:09.005913 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:09.008455   320 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.999559   376 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:09.006521 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:09.056073 30939 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 46216.5us
I20260709 12:24:09.086426 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:08.996192   350 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
W20260709 12:24:09.168973 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 160815us
I20260709 12:24:09.145443   328 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:24:09.029103   356 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.162560 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:09.009071 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:09.006861 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:09.090163 32656 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 12:24:09.247051   356 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:09.179430 31139 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:09.155717   362 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:09.248937   421 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:09.249389   362 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:09.250516   415 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:24:09.168241 32654 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:09.254347 31137 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.255663   384 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:09.256107   384 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:09.256879   384 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:09.258472 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.007135 32660 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.166s
I20260709 12:24:09.196411 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:09.259374 30985 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.180828   382 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:09.260099   318 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:09.260474   318 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.261206   318 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:09.262275 31060 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:24:09.262604 31060 pending_rounds.cc:85] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:24:09.262814 31060 pending_rounds.cc:107] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:09.258009   384 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:09.263087 31060 raft_consensus.cc:2889] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:09.173105 31691 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:24:09.258611 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 53094.9us
W20260709 12:24:09.251874   416 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:09.204737   423 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:09.265188 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:09.272676 30985 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
W20260709 12:24:09.273526 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 111038us
I20260709 12:24:09.172943 32675 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:09.275826 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:09.291940 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:09.199563   394 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:09.306620 31144 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:09.266469   392 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:09.199770   320 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.276759 32716 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:09.313968 31694 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:09.318863   320 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:09.321604   420 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.227s	user 0.002s	sys 0.000s
W20260709 12:24:09.323009   417 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
W20260709 12:24:09.334560   425 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:24:09.340909   318 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.345382 31137 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:09.350112 31693 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:09.354537 31691 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:09.371980 31136 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.378827 31142 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.373199 32660 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:09.361390 32716 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.380388 32727 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:09.382115 31140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:09.403434 31144 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.403561 31140 raft_consensus.cc:606] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:09.404176 31140 raft_consensus.cc:630] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Rejecting request to transfer leadership because tablet server 2c4511d66b474ee0aa41c9b1f03b66ab is not a voter in the active config
I20260709 12:24:09.398193 30985 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.398823 30990 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:09.400352 30991 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:09.411260 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 242012us
W20260709 12:24:09.433892 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 375391us
I20260709 12:24:09.451184 30983 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:24:09.451638 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 45100.9us
I20260709 12:24:09.455683 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:09.454547 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 461774us
I20260709 12:24:09.459682 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:09.463121   376 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:09.466382 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:09.407577 32660 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.522796 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:09.523476 31064 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:09.528084 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 250295us
I20260709 12:24:09.530063 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:09.530263 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:09.531049 30987 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:09.531158 30985 raft_consensus.cc:2368] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3.
I20260709 12:24:09.528465 31060 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:09.538160 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 129269us
W20260709 12:24:09.517128 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57553.2us
W20260709 12:24:09.543709 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 81880.6us
I20260709 12:24:09.542867 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:09.544947 30990 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:09.546924 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:09.451886 31143 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:09.541643 31033 leader_election.cc:400] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5.
I20260709 12:24:09.553395 31033 leader_election.cc:403] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:09.552121 31064 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:09.521538 32656 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:09.526479   392 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:09.559715 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 267862us
I20260709 12:24:09.517894 32675 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.566880 31135 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.568387 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:09.561481 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:09.548655   379 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:09.572913 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:09.570750 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:24:09.572708 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:09.573642 30988 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:24:09.578394 30988 pending_rounds.cc:85] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 4
I20260709 12:24:09.578634 30988 pending_rounds.cc:107] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:09.578846 30988 raft_consensus.cc:2889] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:09.574493 31140 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:09.575528 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:09.575857 31060 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.579689 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:09.523285   350 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:24:09.554932   387 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:09.531863 32654 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.545390   373 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:24:09.553107 32743 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:24:09.580446 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:09.569275 32656 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:09.559974   435 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5.
I20260709 12:24:09.576785   376 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:09.553066   328 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:09.580718 31033 leader_election.cc:400] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3.
I20260709 12:24:09.607429 32656 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.619103 31033 leader_election.cc:403] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:09.586395   377 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:09.583649 32741 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:09.622794 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:09.605099   362 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:09.623056 31141 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:09.605880 31692 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:09.567598   393 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.588781 32675 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.571261 32716 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:09.639724 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.587612   429 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:24:09.623822   304 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3.
I20260709 12:24:09.622416   377 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:09.629925   422 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:24:09.623682   434 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:09.631462   376 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:09.697352 32762 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:09.625972   303 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 12:24:09.707336 32741 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:09.718037 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 173144us
I20260709 12:24:09.698076   362 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.700541   441 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:09.720100 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:09.708164   377 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.695130   320 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:24:09.709358   438 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:09.711004   434 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:09.730173 31033 leader_election.cc:400] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3.
W20260709 12:24:09.731724 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 153722us
I20260709 12:24:09.631953   392 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.742784 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:09.606344 32660 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:09.737076   304 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:09.726751 32762 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:09.731144   438 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.729903   435 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:09.733631   441 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.752172   413 raft_consensus.cc:2243] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:09.729532 32741 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:09.776896 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 201426us
I20260709 12:24:09.776593   413 raft_consensus.cc:2272] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:09.753530   350 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:24:09.764804   435 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:09.765127   438 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:09.781418   450 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:24:09.761503 32721 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:24:09.778504 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:09.778815 32654 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:09.813243   452 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:09.815510   431 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:09.816107   431 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:09.817183   413 ts_tablet_manager.cc:1916] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:09.817031   431 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:09.824409 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 263155us
W20260709 12:24:09.814831   436 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 12:24:09.829758   442 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.128s	user 0.001s	sys 0.001s
I20260709 12:24:09.784513   362 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:09.857364   377 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:09.858296 32744 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:09.860370 31059 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.860555 31060 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:09.862074   419 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:09.862092   356 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:09.862655   419 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.862874   413 ts_tablet_manager.cc:1929] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:09.863056   419 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:09.870846   447 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:09.782096   376 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.831861   435 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.877350   435 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.878274   419 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:09.831125   393 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:09.783136   434 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.866919   413 log.cc:1199] T d50d1465e4b84136bf90de700a93ef2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/wals/d50d1465e4b84136bf90de700a93ef2c
I20260709 12:24:09.879980   434 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:09.880621   413 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51930 (request call id 81) took 1325 ms. Trace:
I20260709 12:24:09.881009   413 rpcz_store.cc:276] 0709 12:24:08.555123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.555365 (+   242us) service_pool.cc:224] Handling call
0709 12:24:08.950661 (+395296us) ts_tablet_manager.cc:1151] Deleting tablet d50d1465e4b84136bf90de700a93ef2c
0709 12:24:08.950759 (+    98us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:24:09.854408 (+903649us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:09.862743 (+  8335us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:09.880544 (+ 17801us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":313703,"tablet-delete.queue_time_us":295254,"thread_start_us":269401,"threads_started":1}
I20260709 12:24:09.831260   441 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:09.862808   356 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.890605 32741 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:09.818624 32762 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.891480 31061 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:09.893239 31068 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:09.889358   384 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.000s
I20260709 12:24:09.892164 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:09.900089 31058 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:09.894272 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 171589us
I20260709 12:24:09.899183 31063 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:09.906404 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:09.907063 31063 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:09.842455   424 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.151s
I20260709 12:24:09.920586 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:09.910627   321 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:24:09.910324   393 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:24:09.930121 31058 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:09.935644 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:09.930994 31063 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:09.937779 32727 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:09.912077 32660 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:24:09.934525   356 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:09.960570 32727 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:10.030371 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:10.024260 32675 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:24:10.065688 32727 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.025763   429 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:10.096647 31062 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:10.053870 31059 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.105656 31064 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:10.024832   386 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:10.096689   419 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:10.112313 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.176050 31135 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:10.181795   444 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:10.182385   444 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:10.190397   444 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.192255   444 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:10.205735 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:10.206763 31144 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:10.208653 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:10.211192 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:10.211766 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.213757   377 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.215206 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1369) took 1268 ms. Trace:
I20260709 12:24:10.211797 31143 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.215565 31141 rpcz_store.cc:276] 0709 12:24:08.947025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.124399 (+177374us) service_pool.cc:224] Handling call
0709 12:24:10.215035 (+1090636us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b540006d6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:10.215153 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1114554112}
I20260709 12:24:10.052706   438 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.223801 31689 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:10.225818 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:10.209017   320 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:10.236222   439 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.236750   439 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.223158 32660 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:10.244796 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.033s	sys 0.079s
W20260709 12:24:10.245867 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.034s	sys 0.080s
I20260709 12:24:10.238637   387 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:10.276954 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 505972us
I20260709 12:24:10.272724 32676 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.257467   434 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.312153 31144 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:10.265997   377 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:10.266340   380 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.305176 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.015s	sys 0.046s
W20260709 12:24:10.357923 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.016s	sys 0.046s
W20260709 12:24:10.358652 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 615918us
I20260709 12:24:10.359575 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:10.373443 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:10.485477 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:10.367007   419 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.372301 32763 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:10.366894   384 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:10.487272   392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.019s	sys 0.045s
W20260709 12:24:10.485491 32716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.005s	sys 0.038s
I20260709 12:24:10.318274 32676 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.373796   429 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.355129   434 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.488176   439 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:10.482872   386 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.401723   432 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.275s
I20260709 12:24:10.371492   380 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:10.515861   392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.019s	sys 0.046s
I20260709 12:24:10.367398   377 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.498973   461 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:10.375298 32730 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.149s
W20260709 12:24:10.524422 32716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.005s	sys 0.038s
I20260709 12:24:10.512486   303 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.526304 32741 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.528074   429 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.534034 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:10.526157 31689 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.540052 31138 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:10.515496   419 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.535553   462 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.556742 30987 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:10.523591 32763 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:10.566411 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.565392   438 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.584280   429 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:10.584944   429 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:10.585803 31059 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:10.587479 31697 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:10.536545   359 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:24:10.571549 32741 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.536962   386 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.585501   429 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:10.595758   455 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:24:10.587806 31137 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.588011   435 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:10.541947   461 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.536137   384 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:10.568663   303 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.629478 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:10.620198   431 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.012s	sys 0.033s
W20260709 12:24:10.635622   460 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:10.637403   461 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.609799   462 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.628703   392 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.638372 31693 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:10.645012   431 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.012s	sys 0.034s
I20260709 12:24:10.640708   419 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:10.641714   380 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:10.628497   457 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:24:10.610046   321 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.651515 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:10.679507 30988 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.646943 32676 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:10.642668   362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.010s	sys 0.037s
W20260709 12:24:10.643182   391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.005s	sys 0.018s
I20260709 12:24:10.675552 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:10.606302 31060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.013s	sys 0.046s
I20260709 12:24:10.643544   435 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.663710   432 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.665253   439 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:10.701145 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:10.709596 31060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.013s	sys 0.047s
W20260709 12:24:10.692459   376 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.036s	sys 0.048s
I20260709 12:24:10.664050   384 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.679008   419 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.710784   391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.006s	sys 0.018s
W20260709 12:24:10.709173   362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.010s	sys 0.037s
W20260709 12:24:10.709230   444 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.005s
I20260709 12:24:10.711081   350 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:24:10.718443 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:10.720201   435 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:10.715495   432 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.720917   429 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.684257   377 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:10.714599   376 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.036s	sys 0.048s
I20260709 12:24:10.694504   434 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:10.755304 31058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.028s	sys 0.040s
W20260709 12:24:10.755661 31058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.028s	sys 0.040s
I20260709 12:24:10.737620   303 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:10.735805 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.024s	sys 0.041s
W20260709 12:24:10.758854 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.024s	sys 0.041s
W20260709 12:24:10.759232 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 735290us
I20260709 12:24:10.715698   439 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:10.722966 32762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.013s	sys 0.017s
I20260709 12:24:10.735430 32763 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:10.738310 31140 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.006s	sys 0.026s
I20260709 12:24:10.714327   320 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.684312   321 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.715857 32741 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.729799 30988 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
W20260709 12:24:10.773211 32762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.013s	sys 0.018s
I20260709 12:24:10.674391   392 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.768332 31062 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:10.829025 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:10.757040 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:10.726857   431 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:24:10.743265 32716 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:10.758708   304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.003s	sys 0.038s
I20260709 12:24:10.735764   419 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:10.796463 31140 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.007s	sys 0.027s
W20260709 12:24:10.797159 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.023s
W20260709 12:24:10.837457 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.010s	sys 0.023s
I20260709 12:24:10.800026   429 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.800275   391 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.808518   321 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.809028 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:10.843858 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:10.810729 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1714) took 1075 ms. Trace:
I20260709 12:24:10.812106   439 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.812610   433 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.812965 32763 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.837675 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:10.814721   464 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:24:10.814443   359 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:10.815188   461 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.866151   461 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.868252   461 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:10.869853 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 360) took 1008 ms. Trace:
I20260709 12:24:10.815603 32743 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:10.870644 32743 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.871188 32743 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.873293 32743 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.873801 32743 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:10.816129 32676 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.874487 32676 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.875813 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:10.816519 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:10.877135 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60720.3us
I20260709 12:24:10.818775 32741 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.806355   320 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:10.820595 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:10.823436   432 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:10.826318 31143 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.000s
I20260709 12:24:10.837199 32716 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.834525 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:10.835335   304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.003s	sys 0.039s
I20260709 12:24:10.840168 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:10.842207 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:10.904513 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:10.845841 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:10.920986 30990 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
W20260709 12:24:10.898104 30957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.045s
I20260709 12:24:10.852455 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:10.921579 30957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.000s	sys 0.045s
I20260709 12:24:10.921931 30985 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
W20260709 12:24:10.855830 32660 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
W20260709 12:24:10.855955   466 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:10.845106   391 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:10.850337 32680 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.003s	sys 0.018s
I20260709 12:24:10.858453 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:10.814059   373 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.870255 31060 rpcz_store.cc:276] 0709 12:24:09.861846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.864474 (+  2628us) service_pool.cc:224] Handling call
0709 12:24:09.893279 (+ 28805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.897244 (+  3965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.897287 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.897783 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.713484 (+815701us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.713539 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.713624 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.869616 (+155992us) raft_consensus.cc:1685] finished
0709 12:24:10.869636 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.869801 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5287,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":1499,"thread_start_us":800404,"threads_started":1,"wal-append.queue_time_us":887845}
W20260709 12:24:10.842908   387 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.009s	sys 0.027s
I20260709 12:24:10.854856   321 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.876459 32676 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.900564 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:10.766754 31063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.039s
I20260709 12:24:10.850677 30987 rpcz_store.cc:276] 0709 12:24:09.735284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.735418 (+   134us) service_pool.cc:224] Handling call
0709 12:24:09.735709 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.573477 (+837768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.810455 (+236978us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b5400225e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:10.810649 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887699200}
I20260709 12:24:10.920172 30988 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:10.864681   462 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:10.928287 31062 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:10.943430 32525 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.946659   384 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:10.844993   429 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.035722   429 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:11.014822   387 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.009s	sys 0.028s
I20260709 12:24:11.036257   429 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.036425   387 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:11.036794   429 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:10.874168 32743 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:10.955083 31141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.022s	sys 0.024s
I20260709 12:24:10.865113 32763 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.014565 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:10.900086 31143 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:10.903288 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.016240   432 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.919884   304 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:24:11.003175 32680 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.003s	sys 0.019s
I20260709 12:24:10.894999 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:10.920569   362 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:10.901409   477 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:10.944651 32660 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.865324   433 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.027313 32525 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:11.022816 31063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.007s	sys 0.039s
I20260709 12:24:11.014066   472 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.043126 31141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.025s	sys 0.024s
I20260709 12:24:10.921451 32741 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.022800   377 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.019716   459 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.016906   373 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.047365 32675 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.042339   468 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:24:11.049156 32743 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.051715 31690 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:11.070952 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.050472   392 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.028220   321 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.076496 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:11.075542 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 194275us
I20260709 12:24:11.083025 30990 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:11.076749   472 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.037364   387 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.093968 32525 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:11.063699 32676 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.099877 32743 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.094895 30988 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:11.063520   431 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:11.088469   459 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.076154 32660 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:11.109265 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.111510 30986 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:11.059263   432 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.085525   377 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:11.089129   419 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.075912   440 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:11.114537 32676 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.094295 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.061647   362 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.127251 32660 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.089167 32675 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.169479   432 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.170171   432 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.170712   432 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.171505   433 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.172739 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.141531   320 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.129961   384 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.144912 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.146278 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:11.058959   477 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.144470   440 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.156112 32763 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:11.157351 32676 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.157033   377 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.158260   391 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:11.104475 32680 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:11.094115   486 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:11.124236 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:11.122956   431 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.183192 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.131443 32741 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:11.198081   476 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:24:11.200491 31139 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:11.180222 32716 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.178915 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:11.170697 32675 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.204336 32554 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:11.229558 32554 consensus_queue.cc:1243] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:11.206928   459 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.124547   304 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.208734   468 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:11.200999   474 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 12:24:11.179982   384 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:11.221464 31137 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 12:24:11.222972   472 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.226202 31065 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:11.231209 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1374) took 1052 ms. Trace:
I20260709 12:24:11.232602   498 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.251984 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:11.202576 32680 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.264240 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.250439 31059 raft_consensus.cc:2410] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:11.264257 31140 rpcz_store.cc:276] 0709 12:24:10.172034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.172184 (+   150us) service_pool.cc:224] Handling call
0709 12:24:10.172472 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.172697 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:10.837064 (+664367us) log.cc:844] Serialized 14 byte log entry
0709 12:24:11.224600 (+387536us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:11.224651 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:11.224693 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.224753 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.224851 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":747565,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":74,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":2591,"thread_start_us":1016198,"threads_started":2}
I20260709 12:24:11.278455   350 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:11.210144   486 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.251788   432 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.249045 31137 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:11.257656   384 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.258563   321 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.252152   468 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.303833 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1476) took 1123 ms. Trace:
I20260709 12:24:11.285912   350 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.302645   477 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.288271   373 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.304210 31063 rpcz_store.cc:276] 0709 12:24:10.180553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.180695 (+   142us) service_pool.cc:224] Handling call
0709 12:24:10.181077 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.193558 (+ 12481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.193603 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.208406 (+ 14803us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.059489 (+851083us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:11.059541 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.059630 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.303635 (+244005us) raft_consensus.cc:1685] finished
0709 12:24:11.303654 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.303780 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13031,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":19321,"thread_start_us":834951,"threads_started":1,"wal-append.queue_time_us":799010}
I20260709 12:24:11.293820   440 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.344025 31136 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:11.368371 31136 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:11.299640 32688 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:11.372028 31136 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.376745 31136 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.299868   498 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.378044 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.331202   350 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.383430 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.367871   434 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.406378 31143 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:11.383034 32688 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.382524   362 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.432297   384 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:11.336155   429 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.455305 31142 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:11.456763   384 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.409278 32660 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:11.462368   493 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:11.462649   498 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.463688   468 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.467613 32688 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.478507   486 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.480028 32680 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.451642   472 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.481053   472 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.457880   431 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.409452   434 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.494536 32741 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.503993 32741 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.523849   434 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:11.565999 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.567224 30987 raft_consensus.cc:2410] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:11.577669 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.583580 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.584985 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.586623 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.587291 30984 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:11.588028 32741 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.589694   484 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:11.598368   350 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:09.883085 30873 catalog_manager.cc:5028] TS 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): tablet d50d1465e4b84136bf90de700a93ef2c (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:24:11.612478 30887 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:11.598675   484 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:11.632477 30873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 1.7494e+06us
I20260709 12:24:11.633502   484 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.634039   419 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:11.634613   419 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.635020 30989 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:11.636557   419 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.624028   504 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.642575   504 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.645273   504 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.614532   384 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.627990 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:11.646458 32554 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:11.646977 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.647966 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:11.648972 32554 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:11.649338 32554 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.649622 32554 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.603389   472 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.643512   453 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.652166   453 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.653103   480 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:11.645306   434 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.656167   434 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:11.658437   488 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.420s	user 0.006s	sys 0.008s
I20260709 12:24:11.658157   303 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.659371   453 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:11.659226   303 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.658133   434 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:11.655074 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:11.662925 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:11.605485 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 438092us
I20260709 12:24:11.674594   384 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.674837 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 547) took 1456 ms. Trace:
I20260709 12:24:11.683466 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 320) took 1316 ms. Trace:
I20260709 12:24:11.683948 31694 rpcz_store.cc:276] 0709 12:24:10.367303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.367437 (+   134us) service_pool.cc:224] Handling call
0709 12:24:10.367713 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.556437 (+188724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.556471 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.556515 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.556542 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.683196 (+1126654us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54006572d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:11.683257 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.683400 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431683456}
I20260709 12:24:11.690179 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:11.695124   477 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:11.700192 31694 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:11.658448   438 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.711789 32554 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.687430   384 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:11.714509 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:11.704650 32655 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:24:11.691045 32763 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:11.717147 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.722374 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.725415 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.710579   433 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:11.693696   453 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:11.675391 31692 rpcz_store.cc:276] 0709 12:24:10.218390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.222360 (+  3970us) service_pool.cc:224] Handling call
0709 12:24:10.242682 (+ 20322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.281612 (+ 38930us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.285569 (+  3957us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.510523 (+224954us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.014427 (+503904us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:11.024305 (+  9878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.033269 (+  8964us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.657776 (+624507us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:11.674446 (+ 16670us) raft_consensus.cc:1685] finished
0709 12:24:11.674478 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.674761 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17409,"raft.queue_time_us":4937,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":31226,"spinlock_wait_cycles":57588480,"thread_start_us":382932,"threads_started":1,"wal-append.queue_time_us":702449}
I20260709 12:24:11.702332   459 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.719074 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 69180.7us
I20260709 12:24:11.693377   504 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:24:11.729230 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.747824 31698 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:11.751139 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.753176 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.755016 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:11.755589 30987 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.757900 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:11.717653 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 54808.2us
I20260709 12:24:11.701824   431 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 12:24:11.723337   387 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.023s	sys 0.049s
I20260709 12:24:11.703676   391 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:24:11.738658 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.739660 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.760248 30986 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:11.735965   433 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.740811   384 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.741302   459 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.775207 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.723658   419 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:11.732676 32763 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.712926   438 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.744282 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.745102   303 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.734977 32655 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.776347 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.761924 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:11.730103 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:11.794342 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.800423 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.801733 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.771826 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.772697 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:11.742082   506 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:24:11.765054   387 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.024s	sys 0.050s
I20260709 12:24:11.776422 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.736912   453 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.778247 30987 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:11.791285 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.810597 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.820299 30984 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:11.812724 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 249) took 1453 ms. Trace:
I20260709 12:24:11.806041 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:11.822901 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 248) took 1504 ms. Trace:
I20260709 12:24:11.820984 31135 rpcz_store.cc:276] 0709 12:24:10.359011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.359222 (+   211us) service_pool.cc:224] Handling call
0709 12:24:10.359600 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.544009 (+184409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.544040 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.558956 (+ 14916us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.046573 (+487617us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:11.046621 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.812295 (+765674us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54003679d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:11.812353 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.812410 (+    57us) raft_consensus.cc:1685] finished
0709 12:24:11.812430 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.812657 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2297,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":409949952,"thread_start_us":439134,"threads_started":1,"wal-append.queue_time_us":631412}
I20260709 12:24:11.823273 31144 rpcz_store.cc:276] 0709 12:24:10.318856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.319003 (+   147us) service_pool.cc:224] Handling call
0709 12:24:10.319401 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.558242 (+238841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.558288 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.567375 (+  9087us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.105106 (+537731us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:11.105256 (+   150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.813542 (+708286us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400063954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:11.813604 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.813667 (+    63us) raft_consensus.cc:1685] finished
0709 12:24:11.813687 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.822844 (+  9157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":560,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":6662,"spinlock_wait_cycles":548317440,"thread_start_us":497809,"threads_started":1,"wal-append.queue_time_us":676959}
W20260709 12:24:11.826388 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 96402.3us
I20260709 12:24:11.795919 31690 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:11.820083 30989 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:11.835150 32554 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.827064   434 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:11.850476 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.850534 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.852101 32763 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.799871 32655 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.855934 32655 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.790097   321 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:24:11.828094 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.857903   321 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:11.859974   453 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.849321   434 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.863056   434 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:11.818317   419 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.852020 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:11.845959 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 84145.8us
I20260709 12:24:11.853894   438 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:11.889446 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.831424   375 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.862025 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.894281 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:11.867977 32727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.070s	sys 0.130s
I20260709 12:24:11.893038 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.896201 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.887848   419 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:11.916323 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.918468 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.922055 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.923544 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.902335   386 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:11.912469 32727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.070s	sys 0.130s
I20260709 12:24:11.912631   514 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:11.942713 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.945956   303 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.946661   303 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.946959   484 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:11.947511   484 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.948731   303 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.927994 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.943468 30990 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:11.950181 32554 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:11.950426 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:11.954121 32554 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.962900 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:11.964102 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.964927 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.965674 30990 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:11.968360 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:11.969079 32554 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.969642   517 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:11.969565 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:11.974668   517 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:11.934615 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:11.975036 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.975848 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.975543   517 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:11.976300 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.976972 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.962307   484 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:11.977551 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:11.977581 31692 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:11.983528 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:11.962567 31693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.034s	sys 0.073s
I20260709 12:24:11.971975   350 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:11.972980 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:11.969086 32741 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:11.991420 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:11.998597 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:11.953313   304 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.990334 31693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.034s	sys 0.074s
I20260709 12:24:11.966751   303 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:11.994603   484 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.004148   384 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:12.002817 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:11.967908   514 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:11.991204   434 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.025281 30990 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:12.017131 32741 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:12.026463 32741 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:12.026747 31138 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1384) took 1429 ms (client timeout 1765 ms). Trace:
I20260709 12:24:12.017455   384 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:12.025316   433 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:12.027412   384 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:12.027274 31138 rpcz_store.cc:269] 0709 12:24:10.597600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.644797 (+ 47197us) service_pool.cc:224] Handling call
0709 12:24:12.026562 (+1381765us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540006d6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:12.026693 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2586844416}
I20260709 12:24:12.028664 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.028908   514 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:12.029256 31138 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:12.029508 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.032738   419 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.026769   350 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:24:12.035173 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.026849 32741 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.036470 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.037161 31143 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 4: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:12.039507 31136 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:12.008404   304 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.042366   434 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:12.044245 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 554) took 1124 ms. Trace:
I20260709 12:24:12.044844 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:12.045408 30987 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.052791 30987 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:12.040877 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.053969 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:12.056021 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 57499us
I20260709 12:24:12.057226 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.057998   489 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.058496   489 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:12.058876 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:12.058930   489 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.045408   304 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.062827 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:12.061769   434 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:12.065007 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:12.071342 31697 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 321) took 1696 ms (client timeout 1641 ms). Trace:
I20260709 12:24:12.061723 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.063212   303 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:12.072885 31697 rpcz_store.cc:269] 0709 12:24:10.373077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.373213 (+   136us) service_pool.cc:224] Handling call
0709 12:24:12.069059 (+1695846us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b540011b7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:12.069176 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490642816}
I20260709 12:24:12.073652 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:12.062745 31143 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:12.062664   419 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.084441 31138 raft_consensus.cc:2410] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:12.077455 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.044622 31693 rpcz_store.cc:276] 0709 12:24:10.919961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.061337 (+141376us) service_pool.cc:224] Handling call
0709 12:24:11.076718 (+ 15381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.115543 (+ 38825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.120233 (+  4690us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.209754 (+ 89521us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.012985 (+803231us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:12.014454 (+  1469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.016712 (+  2258us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.044011 (+ 27299us) raft_consensus.cc:1685] finished
0709 12:24:12.044039 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.044184 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1229,"raft.queue_time_us":92980,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":4984,"thread_start_us":743629,"threads_started":1,"wal-append.queue_time_us":807226}
I20260709 12:24:12.085278 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.074059   434 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:12.072387   304 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:12.086006 31143 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:12.086237   304 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:12.086730   304 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.086501   434 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.087029 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.089782   472 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.090206   472 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.090612   472 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.074353   303 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.093469   303 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:12.085356 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.083614 32741 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.098012 32554 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.097831 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.102340 32554 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.102519 31064 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:12.088115 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.104348 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.106909 31068 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:12.077049 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.122588 32554 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.123577 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.103410   419 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:12.134694 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:12.135283 31064 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.135592 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.136155 31068 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:12.145890 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.147356 30991 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:12.151813 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.153293 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.156623 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.159024 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.159430 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.117457 32741 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.169626 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.170260 32741 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:12.161496 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.133764   419 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:12.175925 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1717) took 1524 ms. Trace:
I20260709 12:24:12.174613 30989 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:12.173710 31064 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:12.179155 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.176326 30983 rpcz_store.cc:276] 0709 12:24:10.651496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.653229 (+  1733us) service_pool.cc:224] Handling call
0709 12:24:10.675737 (+ 22508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.072341 (+396604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.072373 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:11.072412 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:11.072440 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.175724 (+1103284us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54000762d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:12.175766 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.175873 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22144,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8339,"spinlock_wait_cycles":911453056}
I20260709 12:24:12.181519 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.183727 30991 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:12.184966 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.185577 30992 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:12.181830 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.187134 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.187419 31064 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:12.189903 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.135001   431 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:24:12.179939   419 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.202095 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:12.191330 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.186465 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.207569 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.214359 31138 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:12.214536 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.215235 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.215682 31138 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:12.216040 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.216478 31136 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:12.217098 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:12.207906   494 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.969s	user 0.013s	sys 0.000s
I20260709 12:24:12.218487 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 368) took 1449 ms. Trace:
I20260709 12:24:12.218827 31061 rpcz_store.cc:276] 0709 12:24:10.769031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.769187 (+   156us) service_pool.cc:224] Handling call
0709 12:24:10.769558 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.769928 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.769969 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.770569 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.052315 (+281746us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:11.052356 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.052415 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.632967 (+580552us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b4800210258. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef07bdaa1 00007f2ef07b4b37 00007f2ef07b49ca 00007f2ef08ac26f 00007f2ef08abf37 00007f2ef08c055c 00007f2ef08c0508 00007f2ef08c0488 00007f2ef08c0443 00007f2ef08bf6ec 00007f2eef7e9a96
0709 12:24:12.133519 (+500552us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:12.218295 (+ 84776us) raft_consensus.cc:1685] finished
0709 12:24:12.218315 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.218433 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22083,"raft.queue_time_us":1801,"raft.run_cpu_time_us":11985,"raft.run_wall_time_us":860129,"spinlock_wait_cycles":78603904,"thread_start_us":224212,"threads_started":1,"wal-append.queue_time_us":378317}
I20260709 12:24:12.210676   472 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.220386   484 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.220799   484 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:12.221165   484 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.211926 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:12.222627 31068 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:12.226933 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.180886   384 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.243791 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.246812 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.192327   539 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:12.191556 32730 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:24:12.206568 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.259536 31063 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:12.241467   424 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:12.246978   449 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.262590 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.268726 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.270004 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.264334 31061 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:12.212071   375 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:24:12.245954 32654 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:12.273615 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.273942 32654 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.274745 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.274961 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.267354 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.263722 31062 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:12.273216 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 362) took 2322 ms. Trace:
I20260709 12:24:12.278796 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.254799   472 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.281152   537 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:12.281601   375 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:12.281517 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.288163 31136 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:12.276031 32654 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.265419   516 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:12.282568   458 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.808s	user 0.008s	sys 0.016s
I20260709 12:24:12.275902 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:12.285568 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 94321.9us
I20260709 12:24:12.299314 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.277585 31058 rpcz_store.cc:276] 0709 12:24:09.950611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.951431 (+   820us) service_pool.cc:224] Handling call
0709 12:24:09.951771 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.952183 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.952235 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.952691 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.756045 (+803354us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:10.756095 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.756176 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.261024 (+1504848us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b2400159f98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee862d9c4 00007f2ee862bacb 00007f2ee87e6dc9 00007f2ee87e6a55 00007f2ee87e5fb0 00007f2eeb45c488 00007f2eeb45cbe8 00007f2eeb434398 00007f2eeb42f76a 00007f2eeb42ebb6
0709 12:24:12.272909 (+ 11885us) raft_consensus.cc:1685] finished
0709 12:24:12.272929 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.273121 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":3133,"raft.run_wall_time_us":56977,"spinlock_wait_cycles":23540480,"thread_start_us":734337,"threads_started":1,"wal-append.queue_time_us":1032981}
I20260709 12:24:12.286993 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:12.271104   539 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.302459 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.302919 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:12.270013   540 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:12.273613   321 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.301653 32554 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:12.307778 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.314585 32554 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.276891 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.287132   504 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.311182 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.313092 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.320039 31058 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:12.293169   449 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.291666   472 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:12.276752   424 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.315045 32554 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:12.315157 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.315707 31106 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.288724   375 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:12.290884 32655 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:12.313911 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.302484   539 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.332720 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.334182 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.324626 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.326874 31061 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:12.330876 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.339221 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.343225 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.322808   517 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.329128 32655 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.323498   542 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:12.332087   531 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.291407   537 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.337802   391 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.338171   539 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.345999 32654 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:12.346151   380 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:12.375401 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 64284.8us
I20260709 12:24:12.330463   472 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.346937   517 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:12.349005   544 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.427304 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.427944 31142 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:12.428743   489 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.429028 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.319314   540 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.426311 30887 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 4, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 4 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:12.378185   375 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.431384   537 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.431901   540 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.432124   375 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.432575 30887 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 3, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.433408 32654 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:12.430156   384 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.356060 32655 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:12.435590   484 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.437455   484 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.356370 32645 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
W20260709 12:24:12.415253 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 145301us
I20260709 12:24:12.412550 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.424507 30885 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 316) took 3944 ms. Trace:
I20260709 12:24:12.425427 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1756) took 1344 ms. Trace:
I20260709 12:24:12.426276   517 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.434427 32654 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:12.424111   538 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:24:12.425752   419 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.441246 30985 rpcz_store.cc:276] 0709 12:24:11.081463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.081583 (+   120us) service_pool.cc:224] Handling call
0709 12:24:11.081909 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.231963 (+150054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.231991 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:11.232031 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:11.232058 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.425209 (+1193151us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400023fd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:12.425253 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.425373 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344677632}
I20260709 12:24:12.413949 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.466028   380 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:12.475405   380 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:12.476187   419 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.371498   542 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.476861 31059 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:12.464949   377 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:12.464720 32554 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.356437   514 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 12:24:12.469070   304 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.483259   514 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.483661   514 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:12.484157   304 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.437636   303 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.484781   472 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.484434   514 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.485653   514 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.486136   489 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.486545 31060 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:12.463048   424 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.476220   380 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.486011   303 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.487318   380 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.487377 32554 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.491055 32655 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.493067 32655 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:12.440392 30885 rpcz_store.cc:276] 0709 12:24:08.479706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.479841 (+   135us) service_pool.cc:224] Handling call
0709 12:24:08.557544 (+ 77703us) spinlock_profiling.cc:243] Waited 55 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fe090 00007f2eeb89cf3a 00007f2ef1652e9c 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:12.424442 (+3866898us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:09.315433 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:09.335529 (+ 20096us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:09.335575 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:24:09.335601 (+    26us) tablet.cc:662] Decoding operations
0709 12:24:09.336786 (+  1185us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:09.336886 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:09.336926 (+    40us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:24:09.337790 (+   864us) tablet.cc:701] Row locks acquired
0709 12:24:09.337808 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:24:09.337936 (+   128us) write_op.cc:270] Start()
0709 12:24:09.338052 (+   116us) write_op.cc:276] Timestamp: P: 1783599849337891 usec, L: 0
0709 12:24:09.338091 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:24:09.358909 (+ 20818us) log.cc:844] Serialized 3999 byte log entry
0709 12:24:09.730435 (+371526us) op_driver.cc:464] REPLICATION: finished
0709 12:24:09.820035 (+ 89600us) write_op.cc:301] APPLY: starting
0709 12:24:09.820172 (+   137us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:09.820442 (+   270us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:09.820622 (+   180us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:09.833253 (+ 12631us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:09.866872 (+ 33619us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:24:09.866926 (+    54us) write_op.cc:312] APPLY: finished
0709 12:24:09.867261 (+   335us) log.cc:844] Serialized 77 byte log entry
0709 12:24:09.867480 (+   219us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:09.867652 (+   172us) write_op.cc:454] Released schema lock
0709 12:24:09.867704 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:09.868129 (+   425us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:09.868175 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":15,"spinlock_wait_cycles":126486272,"child_traces":[["op",{"apply.queue_time_us":86485,"apply.run_cpu_time_us":4090,"apply.run_wall_time_us":48073,"num_ops":10,"prepare.queue_time_us":208230,"prepare.run_cpu_time_us":4567,"prepare.run_wall_time_us":69580,"replication_time_us":380916,"thread_start_us":111479,"threads_started":2}]]}
I20260709 12:24:12.512691   542 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.516925 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 646) took 1818 ms. Trace:
I20260709 12:24:12.517328 31689 rpcz_store.cc:276] 0709 12:24:10.698547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.029967 (+331420us) service_pool.cc:224] Handling call
0709 12:24:11.034186 (+  4219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.107611 (+ 73425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.114815 (+  7204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:11.122762 (+  7947us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:11.127241 (+  4479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.516682 (+1389441us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b5400512a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:12.516740 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.516864 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146596224}
W20260709 12:24:12.526782 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 86713.3us
W20260709 12:24:12.534734 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 122278us
I20260709 12:24:12.535684 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.543670 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:12.545320 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 237609us
I20260709 12:24:12.547969 30990 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:12.549261 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:12.550880   550 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:12.552999 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.558310 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:12.559024 31061 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:12.561148 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:12.562572 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.564225 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:12.550390 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.568400 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:12.587320 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:12.578495 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1382) took 2342 ms. Trace:
I20260709 12:24:12.585681 31061 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:12.580082 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.603677 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:12.608852 31061 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:12.610096 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.570482 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.540735   375 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.573951 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.605087 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1711) took 1791 ms. Trace:
I20260709 12:24:12.624719 31067 rpcz_store.cc:276] 0709 12:24:10.816526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.853183 (+ 36657us) service_pool.cc:224] Handling call
0709 12:24:10.855407 (+  2224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.089450 (+234043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.089494 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:11.089550 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:11.089593 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.604827 (+1515234us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54001a6dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:12.604875 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.605007 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537857920}
I20260709 12:24:12.606434 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:12.626996 32525 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:12.627254 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.627430 32525 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:12.607179 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:12.544955   419 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.610713 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:12.613267 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44922.3us
I20260709 12:24:12.552634   304 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.602784 31141 rpcz_store.cc:276] 0709 12:24:10.235836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.235949 (+   113us) service_pool.cc:224] Handling call
0709 12:24:10.236252 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.264206 (+ 27954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.264246 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.276313 (+ 12067us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.577841 (+2301528us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b180003b1a0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1
0709 12:24:12.577990 (+   149us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:12.578049 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.578140 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.578196 (+    56us) raft_consensus.cc:1685] finished
0709 12:24:12.578208 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.578419 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27596,"raft.queue_time_us":29749,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":11361,"spinlock_wait_cycles":4043136,"thread_start_us":694464,"threads_started":1,"wal-append.queue_time_us":752789,"wal-append.run_cpu_time_us":3017,"wal-append.run_wall_time_us":1142410}
I20260709 12:24:12.631112 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.560703   542 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.638991 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.640347 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.640388 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.642196 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:12.563525 32655 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.644475 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.563829 32741 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:12.645671 31068 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:12.646370 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.646955 31141 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:12.656606 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:12.657136 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:12.657351 30990 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:12.573624   377 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:12.632591 32525 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.552307   432 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 12:24:12.662683 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:12.604530   303 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.656394 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.659457 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:12.578850   453 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:12.570488   424 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.632671 32743 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.645816 32655 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:12.667228 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.671351 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.632617   304 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:12.653187 31061 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:12.677937 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:12.645147   462 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:12.654554 32645 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:24:12.679401 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.662406   507 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.576566   517 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.660898   542 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.674718 32743 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:12.685777   304 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.675576 32655 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.662921 32741 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.699543   459 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:12.700031 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.704566 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 553) took 2038 ms. Trace:
I20260709 12:24:12.756896 31696 rpcz_store.cc:276] 0709 12:24:10.666338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.666534 (+   196us) service_pool.cc:224] Handling call
0709 12:24:12.069179 (+1402645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.069516 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.069542 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.072255 (+  2713us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.173996 (+101741us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:12.174039 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.174098 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.300821 (+126723us) raft_consensus.cc:1685] finished
0709 12:24:12.300846 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.701259 (+400413us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540011b7d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:12.704486 (+  3227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21882,"raft.queue_time_us":241,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":1529,"spinlock_wait_cycles":3204581760,"thread_start_us":95845,"threads_started":1,"wal-append.queue_time_us":172678}
I20260709 12:24:12.685248 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.697376 31061 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:12.748955 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.762465 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.763309 31058 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:12.764199 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.764281 30888 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.765725 31058 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:12.679351   377 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.643517   375 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.771936 30990 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:12.773576 31141 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:12.783603 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.784250 31691 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:12.675388   303 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.793635 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:12.782122   375 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:24:12.782460   350 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.682761   453 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:12.753347 32743 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.781062 31144 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:12.788484 31067 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:12.802407 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:12.815673 30984 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:12.790262 30888 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.823226 30888 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 3 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.807399   350 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:12.858848   350 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.826469 31058 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:12.861325 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.827674   517 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.862648 31058 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:12.863724 31058 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:12.865351 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:12.867764 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:12.870987 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:12.871796 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.848505 30888 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 3 to 4, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 4 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:12.782460   539 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.878304 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:12.886440 31061 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:12.872975 30888 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.882897 32656 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:12.888592 30888 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.889673 32656 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.893790   303 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:12.899394   539 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:12.906626 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.022s	sys 0.052s
W20260709 12:24:12.907019 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.022s	sys 0.052s
W20260709 12:24:12.907642 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 576795us
I20260709 12:24:12.909401 30888 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.911509 30888 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.914001   542 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.919301 32741 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:12.924111 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.925333 31058 raft_consensus.cc:2410] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:12.892167 32656 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:12.953683 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.955390 31058 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:12.956485 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:12.913443 30888 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.960973 30888 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.964855 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.966435 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:12.966866   516 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:12.967376   516 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:12.967695   362 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.968389   516 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:12.969821   362 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:12.979368   438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.033s	sys 0.037s
W20260709 12:24:12.979812   438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.033s	sys 0.037s
I20260709 12:24:12.984427 30888 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 3, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 3 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.995445   489 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.969910   516 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.996213 30888 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:12.996918 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:12.989259 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:13.000236 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.000082 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:13.000965 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:13.018849   555 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:13.019348 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:13.020900 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:13.022617 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:12.957913 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:12.990547 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 119649us
W20260709 12:24:13.027339 31064 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.019s	sys 0.028s
W20260709 12:24:13.027745 31064 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.020s	sys 0.028s
I20260709 12:24:13.028322 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:13.077466 30986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.010s	sys 0.054s
W20260709 12:24:13.118564 30986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.012s	sys 0.057s
W20260709 12:24:13.083024 31142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.018s	sys 0.028s
W20260709 12:24:13.119740 31142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.022s	sys 0.028s
W20260709 12:24:13.120196 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 131036us
W20260709 12:24:13.082403 31139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.011s	sys 0.006s
I20260709 12:24:13.122141 31058 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:13.122243 31139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.014s	sys 0.008s
I20260709 12:24:13.034014 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:13.128770 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 94845.6us
I20260709 12:24:13.123551 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1407) took 1301 ms. Trace:
I20260709 12:24:13.121176 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:13.136595 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:13.137339 30887 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.023s	sys 0.030s
W20260709 12:24:13.137707 30887 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.024s	sys 0.030s
I20260709 12:24:13.140625 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:13.147578 31065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.020s	sys 0.023s
I20260709 12:24:13.142496 31058 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:13.086287   391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.012s	sys 0.027s
W20260709 12:24:13.086529 31138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.003s	sys 0.028s
I20260709 12:24:13.017395   375 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:13.183223 32554 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:13.032163   439 raft_consensus.cc:2764] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:24:13.190287 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 53764.4us
I20260709 12:24:13.029738 32730 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:13.027360 31143 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:13.030089   377 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:13.186650 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:13.180326 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:13.181488 31065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.024s	sys 0.023s
I20260709 12:24:13.187008 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:13.254160 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:13.030380   384 raft_consensus.cc:2764] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:13.027922   452 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.128125 32727 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:13.256570 31697 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:13.218688 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:13.271876 30988 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.008s
W20260709 12:24:13.414167 30988 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.000s	sys 0.008s
W20260709 12:24:13.203473 31062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.000s	sys 0.007s
W20260709 12:24:13.414839 31062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.000s	sys 0.008s
I20260709 12:24:13.135664   438 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.270783 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:13.217705 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:13.136910   387 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:13.266995   391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.014s	sys 0.034s
W20260709 12:24:13.143929 32654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.007s	sys 0.037s
I20260709 12:24:13.265430 32554 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:13.146920   359 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.009s	sys 0.050s
W20260709 12:24:13.328616 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 74554.1us
W20260709 12:24:13.145067   328 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.016s	sys 0.032s
W20260709 12:24:13.269737 31138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.010s	sys 0.030s
W20260709 12:24:13.349373 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 169144us
W20260709 12:24:13.272395   386 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.004s	sys 0.004s
I20260709 12:24:13.364264 31067 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:13.129267   507 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
I20260709 12:24:13.129056 32608 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.185s
I20260709 12:24:13.259063   555 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:13.219733   356 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:13.157491   554 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:13.379321 31063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.003s	sys 0.011s
W20260709 12:24:13.499454 31063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.004s	sys 0.016s
I20260709 12:24:13.219929   556 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:13.279340 31059 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:13.029887   304 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:13.413770 31136 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:13.529855 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:13.269850   539 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.123s
W20260709 12:24:13.426587 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 155882us
W20260709 12:24:13.414611   393 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.024s	sys 0.044s
W20260709 12:24:13.391942 31137 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.022s	sys 0.035s
W20260709 12:24:13.418044   497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.007s	sys 0.013s
I20260709 12:24:13.277074   375 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:13.187610 32743 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:13.438830 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:13.432124 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.036s	sys 0.067s
I20260709 12:24:13.483758 32554 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:13.594589 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.036s	sys 0.067s
I20260709 12:24:13.429566   553 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:13.440548   328 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.018s	sys 0.033s
W20260709 12:24:13.441519 32654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.009s	sys 0.039s
W20260709 12:24:13.467635   386 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.005s	sys 0.004s
I20260709 12:24:13.469373 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:13.616901 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 147595us
W20260709 12:24:13.440980   359 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.010s	sys 0.059s
W20260709 12:24:13.519399 32655 consensus_meta.cc:287] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.744s	user 0.008s	sys 0.004s
I20260709 12:24:13.536283 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 378) took 1225 ms. Trace:
I20260709 12:24:13.536382 31060 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:13.537587   304 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:13.540416 31693 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:24:13.442524   377 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:13.479668   391 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:13.565490 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:13.509310   448 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:13.536417   565 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:24:13.570712 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:13.665702 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 95098.8us
W20260709 12:24:13.594919 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 961484us
I20260709 12:24:13.477136 32660 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.091s
W20260709 12:24:13.551823   393 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.024s	sys 0.045s
I20260709 12:24:13.615018   539 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:13.575419   497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.007s	sys 0.013s
I20260709 12:24:13.704183 31143 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:13.664811   377 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:13.725781 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:13.726341 32743 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:13.726979 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:13.727295   377 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:13.662107   304 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:13.728549   304 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:13.622234   570 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:24:13.508235   555 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:13.626745 31064 rpcz_store.cc:276] 0709 12:24:12.310854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.314268 (+  3414us) service_pool.cc:224] Handling call
0709 12:24:12.314534 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.314738 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:13.028061 (+713323us) log.cc:844] Serialized 14 byte log entry
0709 12:24:13.535906 (+507845us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:13.535975 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:13.536027 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.536113 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.536229 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26650,"prepare.queue_time_us":41530,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":28276,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":1186157,"threads_started":2}
I20260709 12:24:13.590399   384 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:13.562098 31137 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.022s	sys 0.036s
I20260709 12:24:13.132042 31139 rpcz_store.cc:276] 0709 12:24:11.822682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.216106 (+393424us) service_pool.cc:224] Handling call
0709 12:24:12.231511 (+ 15405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.231690 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:12.356823 (+125133us) log.cc:844] Serialized 14 byte log entry
0709 12:24:13.123089 (+766266us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:13.123210 (+   121us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:13.123264 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.123346 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.123491 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":315212,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":5208,"thread_start_us":871452,"threads_started":2}
I20260709 12:24:13.731200   384 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:13.731475   362 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:13.732651 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:13.733144   384 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:13.733654   384 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:13.733716 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:13.736330   453 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:13.736817 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1745) took 1181 ms. Trace:
I20260709 12:24:13.737845   507 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:13.737180 31065 rpcz_store.cc:276] 0709 12:24:12.555735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.555883 (+   148us) service_pool.cc:224] Handling call
0709 12:24:12.556266 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.556542 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.556581 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.557244 (+   663us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.326849 (+769605us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:13.326905 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.326991 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.736595 (+409604us) raft_consensus.cc:1685] finished
0709 12:24:13.736622 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.736763 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":769260,"threads_started":1,"wal-append.queue_time_us":861508}
I20260709 12:24:13.738209   507 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:13.734153   384 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:13.739030   507 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:13.740147   507 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.740800   384 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:13.620995 32608 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.742762   555 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:13.743412 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1410) took 1459 ms. Trace:
W20260709 12:24:13.744748   564 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.385s	user 0.001s	sys 0.001s
I20260709 12:24:13.743877 31144 rpcz_store.cc:276] 0709 12:24:12.283528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.812798 (+529270us) service_pool.cc:224] Handling call
0709 12:24:12.813159 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.816716 (+  3557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.816757 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.817886 (+  1129us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.210074 (+392188us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:13.210129 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.275193 (+ 65064us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.743170 (+467977us) raft_consensus.cc:1685] finished
0709 12:24:13.743196 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.743347 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2911,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":391744,"threads_started":1,"wal-append.queue_time_us":916361}
I20260709 12:24:13.653115 32655 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:13.652817 31689 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:13.691392   375 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:13.588969   431 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:24:13.695325 32727 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:13.697074   359 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:13.698046   391 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:13.698745 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1753) took 1015 ms. Trace:
I20260709 12:24:13.702522   432 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:13.672717 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:13.703944 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.726946 32743 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:13.753039 32655 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:13.762729   577 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.004s
W20260709 12:24:13.748071 31140 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.251s	user 0.030s	sys 0.025s
I20260709 12:24:13.688369   477 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.733956 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:13.792582 31140 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.031s	sys 0.026s
W20260709 12:24:13.624495   557 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:24:13.746078 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:13.751695 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:13.552269   556 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:13.754366 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 46502.1us
W20260709 12:24:13.754068   516 consensus_meta.cc:287] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.757s	user 0.010s	sys 0.003s
W20260709 12:24:13.773772 31106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.140s	user 0.072s	sys 0.108s
W20260709 12:24:13.795295 31106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.072s	sys 0.108s
I20260709 12:24:13.654829 31693 pending_rounds.cc:85] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 4
W20260709 12:24:13.776100 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.763666 30988 rpcz_store.cc:276] 0709 12:24:12.683400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.683559 (+   159us) service_pool.cc:224] Handling call
0709 12:24:12.683940 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.684180 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.684216 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.689223 (+  5007us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.414542 (+725319us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:13.414579 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.414648 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.698498 (+283850us) raft_consensus.cc:1685] finished
0709 12:24:13.698519 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.698684 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":644095,"threads_started":1,"wal-append.queue_time_us":873209}
W20260709 12:24:13.757714   566 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.146s	user 0.000s	sys 0.004s
I20260709 12:24:13.790482 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:13.729033   438 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:13.794039 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60166.9us
I20260709 12:24:13.755234   539 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:13.824105 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:13.795574 31139 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
W20260709 12:24:13.795707 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 1.16737e+06us
I20260709 12:24:13.763413   384 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.823166 31136 raft_consensus.cc:2410] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:13.800850 31693 pending_rounds.cc:107] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:13.836213 31144 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:13.798316   304 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:13.854458 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:13.861577 31664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.018s	sys 0.032s
W20260709 12:24:13.863437 31664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.018s	sys 0.032s
W20260709 12:24:13.864046 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 514980us
I20260709 12:24:13.865455 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:13.885174 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:13.887349 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:13.858256 31693 raft_consensus.cc:2889] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:13.885675 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:13.858132 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 272) took 1551 ms. Trace:
I20260709 12:24:13.894460 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1411) took 1610 ms. Trace:
I20260709 12:24:13.894929 31135 rpcz_store.cc:276] 0709 12:24:12.284138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.835178 (+551040us) service_pool.cc:224] Handling call
0709 12:24:12.835579 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.835837 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.835873 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.836375 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.272215 (+435840us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:13.272252 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.272314 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.894252 (+621938us) raft_consensus.cc:1685] finished
0709 12:24:13.894271 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.894396 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":435474,"threads_started":1,"wal-append.queue_time_us":830373}
I20260709 12:24:13.899904 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:13.904783 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:13.906420 32554 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:13.928800 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1778) took 1043 ms. Trace:
I20260709 12:24:13.929483 31135 rpcz_store.cc:276] 0709 12:24:12.885623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.927748 (+1042125us) service_pool.cc:224] Handling call
0709 12:24:13.928167 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.928378 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.928415 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:13.928469 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.928530 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.928599 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.928731 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:13.930997 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:13.886682   477 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.890642   480 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:13.972010 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 72204.9us
I20260709 12:24:13.949978 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:13.996867 31135 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 1.
W20260709 12:24:13.987586 32525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.005s	sys 0.010s
W20260709 12:24:14.073467 32525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.006s	sys 0.012s
W20260709 12:24:13.987125 31068 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.009s	sys 0.003s
W20260709 12:24:14.075054 31068 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.011s	sys 0.005s
W20260709 12:24:13.987870 31058 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.000s	sys 0.012s
W20260709 12:24:14.075965 31058 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.001s	sys 0.015s
W20260709 12:24:14.044894 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 68606.1us
W20260709 12:24:14.050593   559 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:24:13.948212   377 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:14.051358   571 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
W20260709 12:24:14.052261   578 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:24:13.910373   423 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:13.939330   446 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:14.118892 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 286) took 1838 ms. Trace:
I20260709 12:24:13.889040   561 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:14.104369 31031 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.008s	sys 0.048s
W20260709 12:24:14.125113 31031 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.008s	sys 0.048s
I20260709 12:24:14.047967 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 340) took 1456 ms. Trace:
I20260709 12:24:14.119338 30986 rpcz_store.cc:276] 0709 12:24:12.280458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.280576 (+   118us) service_pool.cc:224] Handling call
0709 12:24:12.280847 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.337421 (+ 56574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.337451 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.345278 (+  7827us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.118967 (+773689us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.119019 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.119122 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.118694 (+999572us) raft_consensus.cc:1685] finished
0709 12:24:14.118714 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.118837 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4097,"raft.queue_time_us":18331,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7889,"thread_start_us":797632,"threads_started":2,"wal-append.queue_time_us":1076124}
I20260709 12:24:14.077219   438 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:14.109472   359 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.147801   359 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:14.163626 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:14.169749 31060 raft_consensus.cc:2368] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6.
I20260709 12:24:14.156419   446 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:14.179234 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:14.186179   446 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:14.180589 31060 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:14.137856 31693 rpcz_store.cc:276] 0709 12:24:12.545701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.546954 (+  1253us) service_pool.cc:224] Handling call
0709 12:24:13.539647 (+992693us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:13.539918 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.912999 (+373081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.002230 (+ 89231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34936576}
I20260709 12:24:14.199059 31661 leader_election.cc:400] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6.
I20260709 12:24:14.199364 31661 leader_election.cc:403] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:24:14.201150 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:14.194562   350 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:14.204090 31060 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:14.245440 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 495521us
W20260709 12:24:14.253425 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.005s	sys 0.054s
W20260709 12:24:14.253758 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.005s	sys 0.054s
W20260709 12:24:14.251878 31030 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.020s	sys 0.051s
W20260709 12:24:14.259006 31030 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.020s	sys 0.052s
I20260709 12:24:14.260746 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 267) took 2000 ms. Trace:
W20260709 12:24:14.266009 31689 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
W20260709 12:24:14.271377 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 364828us
I20260709 12:24:14.261260 31140 rpcz_store.cc:276] 0709 12:24:12.260108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.496080 (+235972us) service_pool.cc:224] Handling call
0709 12:24:12.496380 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.496577 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:13.792991 (+1296414us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.258319 (+465328us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.258476 (+   157us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:14.258634 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.258728 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.260648 (+  1920us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":76409,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":564,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":17572,"thread_start_us":1755070,"threads_started":2}
I20260709 12:24:14.329895 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:14.330935 31140 raft_consensus.cc:2410] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:24:14.254331 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 815534us
I20260709 12:24:14.331777 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:14.293922 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:14.335681 31067 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:14.314569 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:14.262671   359 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:14.460842 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 129171us
I20260709 12:24:14.461702 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:14.463845 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 171020us
I20260709 12:24:14.464709 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:14.272809   433 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:14.294399   507 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.001s
I20260709 12:24:14.323706 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:14.339305 31060 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:14.481633 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:14.482357 31060 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:14.466151 32525 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:14.333002 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 508961us
I20260709 12:24:14.483893 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:14.486446 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:14.283635   431 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:14.284781   477 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:14.283915 31689 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:14.485278 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 235002us
I20260709 12:24:13.893951 31137 rpcz_store.cc:276] 0709 12:24:12.306352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.880822 (+574470us) service_pool.cc:224] Handling call
0709 12:24:12.881159 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.881394 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:12.881780 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 12:24:13.791737 (+909957us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:13.798422 (+  6685us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:13.800620 (+  2198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.823040 (+ 22420us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.853985 (+ 30945us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38763,"prepare.queue_time_us":184,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":60197,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":6807,"thread_start_us":849228,"threads_started":1,"wal-append.queue_time_us":740729}
I20260709 12:24:14.490166   556 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:14.539930 31032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.010s	sys 0.013s
W20260709 12:24:14.540091 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 56298.2us
W20260709 12:24:14.588498 31032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.011s	sys 0.013s
I20260709 12:24:14.546283 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:14.547068 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 82450.1us
I20260709 12:24:14.552958 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:14.495317   448 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.822s	user 0.007s	sys 0.004s
W20260709 12:24:14.599097 31106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.020s	sys 0.046s
I20260709 12:24:14.599378 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:14.496080 31140 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:14.487311   359 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:14.536211 32525 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:14.607862 32525 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:14.608275 32525 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:14.585906 31068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.035s	sys 0.028s
I20260709 12:24:14.591120 31060 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
W20260709 12:24:14.578998 32727 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.022s	sys 0.034s
I20260709 12:24:14.602073 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1413) took 1349 ms. Trace:
W20260709 12:24:14.599423 31106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.020s	sys 0.046s
I20260709 12:24:14.574872   539 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:14.587899   573 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.267s	user 0.001s	sys 0.001s
I20260709 12:24:14.610531 31060 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:14.578691   356 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:14.579958 32743 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6.
W20260709 12:24:14.625919 31068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.038s	sys 0.030s
I20260709 12:24:14.628579 31060 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:14.639842 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:14.642412 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:14.631947 31140 rpcz_store.cc:276] 0709 12:24:13.252474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.495496 (+1243022us) service_pool.cc:224] Handling call
0709 12:24:14.495846 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.601754 (+105908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.601992 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:14.633972 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.601414 31058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.041s
I20260709 12:24:14.563525   561 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:24:14.541352   477 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:14.649739   477 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:14.650974   477 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:14.651540   477 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:14.653623 31058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.010s	sys 0.042s
I20260709 12:24:14.657143 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 387) took 1662 ms. Trace:
W20260709 12:24:14.607160   568 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.518s	user 0.000s	sys 0.001s
W20260709 12:24:14.661839 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 432627us
I20260709 12:24:14.657687 31068 rpcz_store.cc:276] 0709 12:24:12.994606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.360452 (+365846us) service_pool.cc:224] Handling call
0709 12:24:13.431086 (+ 70634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.431338 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:14.075459 (+644121us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.637785 (+562326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.638358 (+   573us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:14.656817 (+ 18459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.656902 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.657062 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":565344,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":1155281,"threads_started":2}
I20260709 12:24:14.638155 31060 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:14.665546 31060 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:14.668498   558 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.669083   558 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:14.610400 31059 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:14.601375   556 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:14.611763   601 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:14.627511   595 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.575906   431 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:14.606545   448 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:14.680688 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 579) took 1288 ms. Trace:
I20260709 12:24:14.683779 31060 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:14.626451   359 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:14.657338   453 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:14.705780 31696 rpcz_store.cc:276] 0709 12:24:13.392858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.449034 (+ 56176us) service_pool.cc:224] Handling call
0709 12:24:13.473824 (+ 24790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.474063 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:14.043766 (+569703us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.680071 (+636305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.680155 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:14.680269 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.680396 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.680602 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1074700,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":266226,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":926754,"threads_started":3}
I20260709 12:24:14.721210 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 255) took 2478 ms. Trace:
I20260709 12:24:14.721601 31142 rpcz_store.cc:276] 0709 12:24:12.242746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.430031 (+187285us) service_pool.cc:224] Handling call
0709 12:24:12.441140 (+ 11109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.470693 (+ 29553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.470742 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.471305 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.120255 (+648950us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.120308 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.120390 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.720990 (+1600600us) raft_consensus.cc:1685] finished
0709 12:24:14.721011 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.721148 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24516,"raft.queue_time_us":28585,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":600984,"threads_started":1,"wal-append.queue_time_us":1701606}
I20260709 12:24:14.704643   590 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:14.710492 32654 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:14.709347   539 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:14.700310   556 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:14.711836   480 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:14.726866 31137 pending_rounds.cc:85] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 5
I20260709 12:24:14.754230 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 385) took 1760 ms. Trace:
W20260709 12:24:14.760602   574 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:24:14.760973 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1416) took 1506 ms. Trace:
I20260709 12:24:14.771939 31142 rpcz_store.cc:276] 0709 12:24:13.254818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.735647 (+1480829us) service_pool.cc:224] Handling call
0709 12:24:14.760453 (+ 24806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.760629 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.760662 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.760713 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:14.760748 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.760802 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.760924 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.774145 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:14.761183 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 133334us
I20260709 12:24:14.731743   386 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:14.771798 31058 rpcz_store.cc:276] 0709 12:24:12.993928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.276119 (+282191us) service_pool.cc:224] Handling call
0709 12:24:13.362101 (+ 85982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.362311 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:14.076362 (+714051us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.751859 (+675497us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.751930 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:14.753236 (+  1306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.753845 (+   609us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.754144 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3891,"prepare.queue_time_us":530420,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"thread_start_us":1326225,"threads_started":3}
I20260709 12:24:14.779670 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:14.780328 31696 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
W20260709 12:24:14.769017   594 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
W20260709 12:24:14.778709 31698 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 346) took 1566 ms (client timeout 1513 ms). Trace:
W20260709 12:24:14.875234 31698 rpcz_store.cc:269] 0709 12:24:13.212633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.213565 (+   932us) service_pool.cc:224] Handling call
0709 12:24:14.773955 (+1560390us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:14.778631 (+  4676us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4688640}
I20260709 12:24:14.710613   595 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:14.859807 31109 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.151s (SENT) [suppressed 38 similar messages]
I20260709 12:24:14.767465 31137 pending_rounds.cc:107] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 12:24:14.879477 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 96116.4us
I20260709 12:24:14.732198   601 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:14.767421 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 259) took 2517 ms. Trace:
W20260709 12:24:14.877919 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 95099.8us
I20260709 12:24:14.880679 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:14.781373   591 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.885851 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:14.883134   581 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
W20260709 12:24:14.896214   616 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:24:14.883368 31137 raft_consensus.cc:2889] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:14.901053   622 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:14.901598 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1409) took 2618 ms. Trace:
I20260709 12:24:14.901981 31141 rpcz_store.cc:276] 0709 12:24:12.283263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.785200 (+501937us) service_pool.cc:224] Handling call
0709 12:24:12.785472 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.785796 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.785825 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.804798 (+ 18973us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.359588 (+554790us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:13.359626 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.359688 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.901401 (+1541713us) raft_consensus.cc:1685] finished
0709 12:24:14.901423 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.901540 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":263,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":18577,"thread_start_us":488329,"threads_started":1,"wal-append.queue_time_us":1443178}
W20260709 12:24:14.893214   580 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 12:24:14.768261   590 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 12:24:14.884948   600 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:24:14.896250   558 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:14.947475 32727 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:14.894627   591 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:14.967828   623 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:14.967902 32654 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:14.947929   627 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:14.969058   607 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:24:14.956532   600 consensus_queue.cc:1243] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:14.901628   622 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:14.957672 32727 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:14.957397   579 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:14.969139   448 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:24:14.991133   558 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:14.989275   431 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:14.999701   359 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.068322 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1477) took 2449 ms. Trace:
W20260709 12:24:15.055635 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 169859us
W20260709 12:24:15.066748 31030 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.013s	sys 0.021s
W20260709 12:24:15.073665 31030 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.014s	sys 0.021s
I20260709 12:24:15.054563 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1417) took 1461 ms. Trace:
I20260709 12:24:15.075505 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.048136 32727 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:15.077389 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:15.072702 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 212916us
I20260709 12:24:14.884737 31138 rpcz_store.cc:276] 0709 12:24:12.243901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.475667 (+231766us) service_pool.cc:224] Handling call
0709 12:24:12.476065 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.476965 (+   900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.477002 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.506420 (+ 29418us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.489287 (+982867us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.509246 (+ 19959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.532898 (+ 23652us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.733618 (+1200720us) raft_consensus.cc:1685] finished
0709 12:24:14.733650 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.760916 (+ 27266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4045,"raft.queue_time_us":3984,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":2366,"thread_start_us":934062,"threads_started":1,"wal-append.queue_time_us":1257051}
I20260709 12:24:15.083781 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:15.084010   603 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:15.084468   603 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:15.084834   603 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.084471 31058 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
W20260709 12:24:15.085778   585 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.271s	user 0.001s	sys 0.001s
W20260709 12:24:15.087065   438 consensus_meta.cc:287] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.930s	user 0.010s	sys 0.000s
I20260709 12:24:15.087464   438 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:14.995889   627 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.072374   599 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:15.074573   592 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:15.103736 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.073503 31063 rpcz_store.cc:276] 0709 12:24:12.618584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.622775 (+  4191us) service_pool.cc:224] Handling call
0709 12:24:12.623050 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.623407 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.623434 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.647108 (+ 23674us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.499825 (+852717us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.499863 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.499925 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.004125 (+1504200us) raft_consensus.cc:1685] finished
0709 12:24:15.066166 (+ 62041us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.067814 (+  1648us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3941,"raft.queue_time_us":80,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":15003,"thread_start_us":837780,"threads_started":1,"wal-append.queue_time_us":1660252}
W20260709 12:24:15.104754 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.095407 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.100507 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.106786 31063 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:24:15.082387   453 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:14.999588   623 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.126209   623 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.126767   623 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:15.091243   605 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:24:15.123447 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1770) took 2249 ms. Trace:
W20260709 12:24:15.125231 32525 consensus_meta.cc:287] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.516s	user 0.015s	sys 0.006s
I20260709 12:24:15.132869 32525 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.075897 31142 rpcz_store.cc:276] 0709 12:24:13.593345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.773414 (+1180069us) service_pool.cc:224] Handling call
0709 12:24:14.878900 (+105486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.879139 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:14.889233 (+ 10094us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.051160 (+161927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.052783 (+  1623us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:15.053175 (+   392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.053946 (+   771us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.054493 (+   547us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20317,"prepare.run_cpu_time_us":228,"prepare.run_wall_time_us":14915,"raft.queue_time_us":175,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":18723,"thread_start_us":157647,"threads_started":1,"wal-append.queue_time_us":229302}
W20260709 12:24:15.141022 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.027s	sys 0.010s
W20260709 12:24:15.141429 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.027s	sys 0.010s
W20260709 12:24:15.141733 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 678489us
I20260709 12:24:15.130317 31136 rpcz_store.cc:276] 0709 12:24:12.873601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.835616 (+962015us) service_pool.cc:224] Handling call
0709 12:24:13.856652 (+ 21036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.856917 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.856953 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.884619 (+ 27666us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.288276 (+403657us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:14.288312 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.288387 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.123238 (+834851us) raft_consensus.cc:1685] finished
0709 12:24:15.123258 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.123388 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":214226,"threads_started":1,"wal-append.queue_time_us":693482}
I20260709 12:24:15.154510 31058 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:15.157663 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.092651   599 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:15.073951   558 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.100194 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 348) took 1886 ms. Trace:
I20260709 12:24:15.118686   556 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.170059 32525 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.069837   579 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.148762   356 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:15.128930   453 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:15.183809   636 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:15.184582 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.149413   452 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:15.189306 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 275) took 1281 ms. Trace:
I20260709 12:24:15.177295 31691 rpcz_store.cc:276] 0709 12:24:13.213288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.219270 (+  5982us) service_pool.cc:224] Handling call
0709 12:24:15.093997 (+1874727us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fe9c 00007f2ee876fa5b 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:15.094183 (+   186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.099814 (+  5631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.099848 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.099891 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:15.099942 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.100011 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.100133 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13834368}
I20260709 12:24:15.196516 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:15.236178   632 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:15.236301 31698 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:24:15.236949 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1414) took 1984 ms. Trace:
I20260709 12:24:15.207617   599 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.237370 31137 rpcz_store.cc:276] 0709 12:24:13.252798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.625487 (+1372689us) service_pool.cc:224] Handling call
0709 12:24:14.710006 (+ 84519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.901432 (+191426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.901477 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.907370 (+  5893us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.173186 (+265816us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:15.178516 (+  5330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.182067 (+  3551us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.236721 (+ 54654us) raft_consensus.cc:1685] finished
0709 12:24:15.236741 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.236884 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":204244,"threads_started":1,"wal-append.queue_time_us":270017}
I20260709 12:24:15.243628 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1769) took 2370 ms. Trace:
I20260709 12:24:15.244017 31139 rpcz_store.cc:276] 0709 12:24:12.873331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.835199 (+961868us) service_pool.cc:224] Handling call
0709 12:24:13.854086 (+ 18887us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.889851 (+ 35765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.889900 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.891623 (+  1723us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.498335 (+606712us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:14.498414 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.498583 (+   169us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.243397 (+744814us) raft_consensus.cc:1685] finished
0709 12:24:15.243423 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.243563 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28771,"raft.queue_time_us":31516,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10562,"thread_start_us":393012,"threads_started":1,"wal-append.queue_time_us":1262594}
I20260709 12:24:15.247213   579 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:15.250337   558 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:15.253721   350 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.031s	user 0.011s	sys 0.000s
W20260709 12:24:15.255040   602 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:15.255198   377 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.255515   627 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.255689   377 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.258177 32727 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:15.259804 31059 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.000s
I20260709 12:24:15.259919   627 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:15.260196 31059 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:15.260612   622 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.260713   452 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.261471 32727 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.262594 32727 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:24:15.263064   554 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 12:24:15.260756 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 76339.4us
I20260709 12:24:15.264479 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.265164   635 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:15.191382   591 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.181419   356 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:15.183131   601 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.096995   592 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:15.216872 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 121862us
I20260709 12:24:15.205144   387 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:15.227020 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:15.213961   595 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.234612 32525 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.367487 32525 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.367914 30988 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:15.233810 31138 rpcz_store.cc:276] 0709 12:24:13.907583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.084312 (+1176729us) service_pool.cc:224] Handling call
0709 12:24:15.084653 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.084866 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:15.085164 (+   298us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.187798 (+102634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.188070 (+   272us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:15.188294 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.188586 (+   292us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.189095 (+   509us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1779,"prepare.queue_time_us":814,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":10404,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":89296,"threads_started":1,"wal-append.queue_time_us":152640}
I20260709 12:24:15.194186   453 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.260630   597 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.381009 32525 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:15.266448 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.271631   423 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:15.272182 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.278081 31137 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:15.279120 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.361505   387 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.292358   384 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:15.293371 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:15.362775   603 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.364080   640 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:15.366005 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.366693 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.366961 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:15.409037 30990 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:15.409619 30983 raft_consensus.cc:2410] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:24:15.382313 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 115934us
W20260709 12:24:15.394975 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 101683us
I20260709 12:24:15.395076 32525 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:15.410712 32525 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:15.396479 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.411989 31144 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:15.412632 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1772) took 2538 ms. Trace:
I20260709 12:24:15.412911 31144 rpcz_store.cc:276] 0709 12:24:12.874110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.855818 (+981708us) service_pool.cc:224] Handling call
0709 12:24:15.411632 (+1555814us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:15.411806 (+   174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.412406 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.412576 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2827520}
I20260709 12:24:15.263044   377 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.402988 30989 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:15.265746   635 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.379261   431 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:15.411468 32525 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.421101 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.415650 30990 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:24:15.391390   591 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:15.410897 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:15.422605 30990 pending_rounds.cc:85] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 1
I20260709 12:24:15.425933 30991 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.427608 30990 pending_rounds.cc:107] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:24:15.433578 30990 raft_consensus.cc:2889] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:15.436774 31136 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:15.437935 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.444388 31696 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:15.445878 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.443866 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:15.405246   649 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:15.405402   640 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:15.453621 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:15.460245 31063 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:15.395853   423 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:15.456146 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.456318 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.460310 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 274) took 1553 ms. Trace:
I20260709 12:24:15.466516 31141 rpcz_store.cc:276] 0709 12:24:13.907262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.904691 (+997429us) service_pool.cc:224] Handling call
0709 12:24:14.905002 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.905374 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.905411 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.923188 (+ 17777us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.124695 (+201507us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:15.124767 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.124860 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.460055 (+335195us) raft_consensus.cc:1685] finished
0709 12:24:15.460083 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.460228 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10278,"raft.queue_time_us":956,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":21839,"thread_start_us":143001,"threads_started":1,"wal-append.queue_time_us":308629}
I20260709 12:24:15.424177   384 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.472213 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:15.440860 31033 connection.cc:256] RPC call timeout handler was delayed by -1.2814392280001812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 12:24:15.472908 31033 connection.cc:256] RPC call timeout handler was delayed by -1.250436807999904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
I20260709 12:24:15.450452   350 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
W20260709 12:24:15.473284 31033 connection.cc:256] RPC call timeout handler was delayed by -1.223839228000088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
I20260709 12:24:15.473346   603 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:15.473719 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0440264740000202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 12:24:15.474035 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0440264740000202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 12:24:15.474344 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0231392280002183s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
I20260709 12:24:15.474360   591 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:15.475044   591 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.476111   637 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:15.476943   591 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.431596 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:15.458740   606 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:15.468780 31060 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:15.478681   606 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.479399 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:15.479688 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:15.480115 31060 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:15.480352 31068 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:15.480881 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:15.481374   606 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.481748 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:15.482901 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.458726   579 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:15.484017   579 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:15.484102   553 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:15.484596   553 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.484944   304 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:15.485353   304 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:15.485745   304 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.486198   635 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.443790   604 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:24:15.487120   604 consensus_queue.cc:1243] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:15.489234   516 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:15.465565   649 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.489655   516 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:15.490468   516 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.491051 31058 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:15.481374 31068 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.491398 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.492240 30992 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 1.
I20260709 12:24:15.492741 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 299) took 1216 ms. Trace:
I20260709 12:24:15.493034 30992 rpcz_store.cc:276] 0709 12:24:14.276221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.276319 (+    98us) service_pool.cc:224] Handling call
0709 12:24:15.491218 (+1214899us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:15.492686 (+  1468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22801664}
I20260709 12:24:15.493530 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.469734 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.455116 32655 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:24:15.497804   628 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:15.393859   597 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.472532   384 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.440371   554 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:15.424304   624 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:15.468003 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:15.503175 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:15.505362 31063 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.517503 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:15.498210   628 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:15.511766 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.511255 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:15.520088 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:15.455266   452 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.445408   431 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.527796 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:15.528034 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60111.5us
I20260709 12:24:15.529031   500 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:15.529264 30992 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:15.536293 30992 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:15.514114 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:15.531283 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:15.408747   356 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.528820 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:15.512679   638 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:15.517231   477 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.521183   603 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:15.537050 30992 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.525239   554 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:15.542348 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:15.560251 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:15.560979 31068 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:15.555518 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:15.562641 31067 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:15.556610 31058 pending_rounds.cc:85] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 2
I20260709 12:24:15.563580 31058 pending_rounds.cc:107] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:24:15.563913 31058 raft_consensus.cc:2889] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:15.531909   462 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:24:15.565598   631 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:15.566042   631 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:15.566458   631 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:15.534868 32654 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:15.571425 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.574774   625 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:15.575418   625 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:15.575992   621 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:24:15.576699   628 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.578002   628 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.578733 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.580034 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.557056 31691 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:15.583144 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:15.583877 31691 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 1.
I20260709 12:24:15.584100   553 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.584992 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:15.484966   579 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.586808   377 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:24:15.572678   603 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.552470   500 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.526829   424 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:24:15.529270   549 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:15.585727 31691 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:15.561841   640 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.618594   655 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.619380   640 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.619138   655 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.621358   655 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.558707   623 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:15.573439 32654 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.573652 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:15.630472 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75671.2us
I20260709 12:24:15.622280   553 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.632834   657 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.633389 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:15.590283 32727 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:15.520907   624 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.616945   356 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.613340   500 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:15.592129   387 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.630545 32654 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.639477 32727 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:15.636044   592 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.622970   477 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:15.633996   653 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:24:15.634335   649 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.637696   623 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.643890   350 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:24:15.642412   480 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:24:15.657519   603 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.659147   431 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.642486   549 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.677357   480 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:24:15.679422   569 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 2.053s	user 0.002s	sys 0.003s
I20260709 12:24:15.679544   480 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:15.680507   480 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.681764   480 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:15.639868   657 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.671562   384 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.683585   549 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.683290   631 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.684392   657 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.685794   631 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.662740   477 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:15.692426   650 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.166s	user 0.001s	sys 0.001s
I20260709 12:24:15.652554   554 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:15.716761   658 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
W20260709 12:24:15.716895   654 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:24:15.762404 30987 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
W20260709 12:24:15.765883 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.771189 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 242296us
I20260709 12:24:15.772845 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.775014 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:15.777094 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.787855 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 376) took 3477 ms. Trace:
I20260709 12:24:15.788507 31062 rpcz_store.cc:276] 0709 12:24:12.310262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.317399 (+  7137us) service_pool.cc:224] Handling call
0709 12:24:12.317735 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.531644 (+213909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.531692 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.539072 (+  7380us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.415152 (+876080us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.415188 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.415248 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.444044 (+2028796us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:15.780156 (+336112us) raft_consensus.cc:1685] finished
0709 12:24:15.780203 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.787620 (+  7417us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1337,"raft.queue_time_us":19430,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":2487,"spinlock_wait_cycles":14228864,"thread_start_us":1087823,"threads_started":2,"wal-append.queue_time_us":997514}
W20260709 12:24:15.793217 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 249765us
I20260709 12:24:15.796861 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:15.642138 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:15.801247 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.801535 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 159399us
I20260709 12:24:15.810738 31142 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:15.814076 31143 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.002s
I20260709 12:24:15.814960 31143 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:15.815845 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.818401 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:15.820513 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:15.822124 31689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 665) took 3150 ms (client timeout 2151 ms). Trace:
I20260709 12:24:15.836030   387 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:15.839542   477 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:15.853901 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59726.1us
I20260709 12:24:15.858019 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:15.858666   631 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.859278   657 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:15.860313   657 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:15.859694   631 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.880616   387 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:15.881191   665 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:15.881645 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 293) took 1607 ms. Trace:
I20260709 12:24:15.881820   665 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:15.882507   665 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.868288   553 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:15.883769   553 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.884855 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.882401 32655 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:15.888929   606 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.889757   606 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.890713 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:15.891846 31062 raft_consensus.cc:2393] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:15.892882 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.898679 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.893884 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.901304 30990 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:15.901588   622 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:15.902872   622 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:15.903538   622 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:15.676326 32727 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.905750 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:15.886691 30986 rpcz_store.cc:276] 0709 12:24:14.273983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.274195 (+   212us) service_pool.cc:224] Handling call
0709 12:24:15.879934 (+1605739us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:15.880483 (+   549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.880936 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.880988 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.881066 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:15.881105 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.881170 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.881368 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27719424}
I20260709 12:24:15.915678   453 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.905580   553 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.917299 31144 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:15.922838 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 102596us
I20260709 12:24:15.924993   604 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.907932 32727 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.925767   604 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:15.928297   604 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:15.911849 31140 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:24:15.848003 31689 rpcz_store.cc:269] 0709 12:24:12.671445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.674803 (+  3358us) service_pool.cc:224] Handling call
0709 12:24:15.821553 (+3146750us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b540081e7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:15.821801 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164983040}
I20260709 12:24:15.948633 31691 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:15.942385 32525 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:15.949045   624 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:15.949839 32525 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:15.950253 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1415) took 2687 ms. Trace:
I20260709 12:24:15.956899   630 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:15.878116   666 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:15.962999   664 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:15.938552   631 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:15.954939 32525 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:15.965546 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.955509 31140 rpcz_store.cc:276] 0709 12:24:13.253068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.646974 (+1393906us) service_pool.cc:224] Handling call
0709 12:24:15.910531 (+1263557us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:15.911065 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.939150 (+ 28085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.940648 (+  1498us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30245120}
I20260709 12:24:15.974592   350 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:15.975942   350 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:15.976363   350 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:15.973420   500 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:15.954589   453 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:15.982568 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:15.981999 31062 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.983038 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 574) took 2731 ms. Trace:
I20260709 12:24:15.985435 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:15.988569 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 578) took 2596 ms. Trace:
I20260709 12:24:15.993105 31698 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:15.993208 31693 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 1.
I20260709 12:24:15.990859 31692 rpcz_store.cc:276] 0709 12:24:13.251304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.266179 (+ 14875us) service_pool.cc:224] Handling call
0709 12:24:15.981667 (+2715488us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:15.982103 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.982531 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.982566 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.982614 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:15.982674 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.982733 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.982837 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20041600}
I20260709 12:24:15.997277   375 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:15.997815   375 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:15.999511 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.999138   375 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.000558   375 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.996274 32654 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.002260 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:16.003430   433 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:16.004004   433 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.905287   622 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.012049 31698 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:15.991104 31697 rpcz_store.cc:276] 0709 12:24:13.392293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.414137 (+ 21844us) service_pool.cc:224] Handling call
0709 12:24:13.414473 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.743421 (+328948us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:14.251006 (+507585us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.725426 (+474420us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.725484 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:14.725556 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.988134 (+1262578us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b540076c354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:15.988194 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.988370 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":805617,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":81,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":16260,"spinlock_wait_cycles":756043648,"thread_start_us":973583,"threads_started":2,"wal-append.queue_time_us":930163}
I20260709 12:24:16.025522 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:16.031445   304 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.005s
I20260709 12:24:16.031955 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:16.032171   304 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.033711 30987 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:16.033957 31064 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:16.035503 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:16.037904   304 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.053072 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 581) took 1798 ms. Trace:
I20260709 12:24:16.053534 31690 rpcz_store.cc:276] 0709 12:24:14.249479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.251243 (+  1764us) service_pool.cc:224] Handling call
0709 12:24:15.475521 (+1224278us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876feee 000000000036eb39 00007f2ef07e31e0 00007f2ef08ac0c4 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:15.975983 (+500462us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876ff8c 00007f2ee876fef6 000000000036eb39 00007f2ef07e31e0 00007f2ef08ac0c4 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:15.976287 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.976788 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.976819 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:16.015424 (+ 38605us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:16.019980 (+  4556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.027071 (+  7091us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.048261 (+ 21190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112970368}
I20260709 12:24:16.054616 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:16.055379 31692 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:16.062320 31697 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
W20260709 12:24:16.065198 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 51304us
I20260709 12:24:16.069518 32525 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.073612 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:16.067752   604 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:16.077051   604 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.077859 32730 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:16.078608 32730 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.080137   604 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.080948 32730 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:16.081415 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.050396   606 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:16.085515   356 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:16.086539   356 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.088403   549 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:16.088791   549 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:16.089164   549 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:16.090117 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:16.091156 30983 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:16.091849   393 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:16.098842 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.100706 30983 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:16.098055 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.104486 31698 raft_consensus.cc:2410] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:16.104539 32525 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:16.110567   518 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:16.110699 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.111495 31135 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:16.111328   518 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.113655   518 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.114351 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:16.115509   446 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.115999   446 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.116426   446 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.117496   386 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:16.118043 32525 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:16.118310 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:16.119305   453 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:16.119824   453 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:16.120437   423 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:16.121109   500 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:16.122026 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.122766 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:16.123587 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.121155   453 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.125685 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.126176   453 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:16.126724 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.125092 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:16.125167 31059 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.125397 31062 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:16.131297 32721 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:16.132144 32721 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:16.133180 32721 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:16.133567 32721 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:16.134984 32721 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.136251 32721 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.136487   350 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.100149 32654 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:16.138796 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:16.123608 31063 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:16.117494 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.099627   606 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:24:16.145820 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:16.166458 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 64400.7us
I20260709 12:24:16.167582 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.118070   386 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:16.138172   350 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.176525   433 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.139688   555 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:16.163242 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.171381   549 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:16.178635 30985 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 298) took 1902 ms (client timeout 1627 ms). Trace:
W20260709 12:24:16.185386 30985 rpcz_store.cc:269] 0709 12:24:14.275866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.276003 (+   137us) service_pool.cc:224] Handling call
0709 12:24:16.145451 (+1869448us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:16.178505 (+ 33054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23763840}
I20260709 12:24:16.192668 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:16.193408 30984 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.183231   555 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:16.196548   433 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.196210   555 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.199131   555 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.201290   386 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.203887 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.204378   356 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:16.207825   549 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.209568 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 572) took 2958 ms. Trace:
I20260709 12:24:16.174041 31692 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:16.209951 31695 rpcz_store.cc:276] 0709 12:24:13.250655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.251483 (+   828us) service_pool.cc:224] Handling call
0709 12:24:13.251858 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.081583 (+1829725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.081615 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.123405 (+ 41790us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.227313 (+103908us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:15.227356 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.163238 (+935882us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54006a1654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:16.165937 (+  2699us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.179095 (+ 13158us) raft_consensus.cc:1685] finished
0709 12:24:16.179118 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.209400 (+ 30282us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2856,"raft.queue_time_us":2979,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4615,"spinlock_wait_cycles":4200516864,"thread_start_us":47981,"threads_started":1,"wal-append.queue_time_us":330198}
I20260709 12:24:16.211364 30984 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:16.213440 31135 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
W20260709 12:24:16.215350 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68277.5us
I20260709 12:24:16.215046 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.217522   597 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.220813 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.222316   600 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:16.222859   600 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:16.178133 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.230306 30989 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:16.217998   350 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:16.248807 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 70938.7us
I20260709 12:24:16.256147 31063 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:16.238318   356 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:16.258064 32727 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:16.258816   680 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.124s	user 0.002s	sys 0.000s
I20260709 12:24:16.259030 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.251024   350 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:16.224009   600 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:16.265349 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.263978 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.274449   623 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.251860   423 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:16.278626   423 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.281186   423 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:16.283434 31143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1775) took 3400 ms (client timeout 2113 ms). Trace:
I20260709 12:24:16.284116   350 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:16.283885 31143 rpcz_store.cc:269] 0709 12:24:12.883164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.726740 (+843576us) service_pool.cc:224] Handling call
0709 12:24:16.283203 (+2556463us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540001a754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:16.283335 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3511246208}
I20260709 12:24:16.287221 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:16.288889 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.288769 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.289564 31140 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:16.291589 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.292500 31140 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:16.293932 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.272075   356 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:16.295763   356 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.296167 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:16.296977 30984 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.298852 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:16.260633 32727 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:16.309091   375 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.313681 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.318035 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:16.314316 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.326148 30984 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:16.331156 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:16.331885 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.328658 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:16.322187 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.308808 32727 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:16.334424 31140 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:16.327340 32730 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:16.335326 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:16.307366   665 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.315858   375 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.338785 32730 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:16.355181 32730 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:16.357033   610 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.408s	user 0.004s	sys 0.000s
I20260709 12:24:16.339646   386 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:16.340070 31139 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:16.343752 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:16.345628   685 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
I20260709 12:24:16.347754 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:16.338198   555 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:16.384052 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.365362 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:16.391907 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:16.394042 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:16.396867 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:16.407985 31695 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:16.395916   555 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.385417   446 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.424775 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:16.405972 31137 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:16.416625 32608 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:24:16.435518 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 123305us
I20260709 12:24:16.423540 31692 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:16.444339 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:16.446659 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:16.448040 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 61725.5us
I20260709 12:24:16.446743 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:16.453059 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:16.456161 30983 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:16.458398 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:16.469780 30985 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:16.455072   375 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:16.473729   665 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:16.443274   438 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:24:16.478358 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:16.481325 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:16.486362   622 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:16.486995 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 576) took 3225 ms. Trace:
I20260709 12:24:16.487246 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:16.487401 31694 rpcz_store.cc:276] 0709 12:24:13.261097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.261296 (+   199us) service_pool.cc:224] Handling call
0709 12:24:13.261664 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.625703 (+364039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.625733 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.626156 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.826464 (+200308us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.826510 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.486416 (+2659906us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b540076ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:16.486500 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.486564 (+    64us) raft_consensus.cc:1685] finished
0709 12:24:16.486577 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.486791 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836561664,"thread_start_us":188796,"threads_started":1,"wal-append.queue_time_us":654877,"wal-append.run_cpu_time_us":7792,"wal-append.run_wall_time_us":1611557}
I20260709 12:24:16.488920   623 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:16.489362   623 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:16.489488 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.490319 31698 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:16.488960 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.492422 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.494539 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:16.495203   446 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:16.496168 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:16.497658 31058 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.490357 31067 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:16.452899 32608 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.497670 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.505764 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.510233 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.511363 31067 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:16.512054 30986 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:16.513833 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.511933 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.517280 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:16.518458 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 71124.6us
I20260709 12:24:16.521378 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.522066 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.521291 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:16.523785 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:16.524972 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.525208 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:16.515129   665 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.527455 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:16.528789 30986 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.487023   622 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.529925 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:16.532742 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.532043 31060 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:16.532817 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.533718 31058 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:16.537714   597 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.529464   555 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:16.537758 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:16.538501 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.525143 32608 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:16.528342 31692 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:16.542428   438 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:16.542601 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.542920   438 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:16.543087   682 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.425s	user 0.000s	sys 0.003s
I20260709 12:24:16.543361   438 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:16.543963   600 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.545396 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.546350   362 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.546783   362 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:16.547168   362 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.548374 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:16.550410 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:16.550671 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.505511   623 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:16.551374 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.551929 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.553866   375 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:16.554041 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.554337 31065 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:16.554513   507 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:16.554965   507 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.555305   507 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:16.555332 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.555878 31067 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:16.554348   375 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:16.556963   375 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:16.558799 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.559268 31059 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:24:16.559841   697 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:24:16.555347 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:16.563947 31065 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.564944 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:16.534155 32727 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.569916   623 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:16.573320 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:16.574806 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.570379 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:16.576088   597 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:16.570762   599 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.579563 30955 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:16.581035 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.581806   667 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.582577   667 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:16.584488 31061 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.566015   600 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:16.583514 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.586119 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:16.571697 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.587563 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.589192 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.590962   622 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.592178 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.593508 32645 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:16.594106 32645 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.585520   600 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:16.595906   600 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:16.596460 32645 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.603168   362 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.603835 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1421) took 1324 ms. Trace:
W20260709 12:24:16.614497 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48376.2us
I20260709 12:24:16.614454 31142 rpcz_store.cc:276] 0709 12:24:15.279833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.279952 (+   119us) service_pool.cc:224] Handling call
0709 12:24:15.280294 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.811320 (+531026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.603569 (+792249us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540012ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:16.603772 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218222848}
I20260709 12:24:16.605926 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:16.607059 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.616307 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:16.616755   606 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.617547 30985 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:16.618590 32727 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:16.610337 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 43787.5us
I20260709 12:24:16.619299 32721 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:16.626461 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.608359 30992 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:16.620520 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1427) took 1339 ms. Trace:
I20260709 12:24:16.626523 32721 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:16.629952 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.627316 31144 rpcz_store.cc:276] 0709 12:24:15.281458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.422008 (+140550us) service_pool.cc:224] Handling call
0709 12:24:15.422316 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.917718 (+495402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.620209 (+702491us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b540006f9d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:16.620446 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137479552}
I20260709 12:24:16.635428 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:16.640399 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:16.644714 31694 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:16.653895 32525 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:16.631359   595 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.636619   453 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:16.665258 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.616334   362 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:16.669263 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:16.646311 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:16.647629 32727 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:16.672679 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 89303.1us
I20260709 12:24:16.656500   622 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:16.681730 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 53693us
I20260709 12:24:16.645118   604 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:24:16.687005 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:16.689282   432 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:16.689359   362 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:16.690027   432 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:16.690632   362 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:16.691098   362 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:16.691257   432 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.691834   579 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.694139   627 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:16.694556   627 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.675616   667 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:16.694912   627 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:16.696465 30955 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.676981 32727 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:16.736508 32727 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:16.741349 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.682732   453 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.697894   667 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:16.749464 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 53114.1us
W20260709 12:24:16.748894 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 65199.2us
I20260709 12:24:16.750190 30955 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.749889 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:16.751463 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 95367.5us
I20260709 12:24:16.753820 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 404) took 1246 ms. Trace:
I20260709 12:24:16.754707 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:16.757413 32525 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.754266 31064 rpcz_store.cc:276] 0709 12:24:15.502453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.503994 (+  1541us) service_pool.cc:224] Handling call
0709 12:24:15.504330 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.034691 (+530361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.745657 (+710966us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54002a4c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:16.748856 (+  3199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1216984960}
I20260709 12:24:16.768222   516 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:16.770427 32654 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:16.770994 32654 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.771885 31065 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:16.773229 32654 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:16.731568   597 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:24:16.774713   597 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:16.775046   597 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:16.776005   597 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.777191   597 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:16.753942   702 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:24:16.751940   667 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.753559   555 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.805214   655 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:16.805346   652 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:24:16.810182   652 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:16.810762   652 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:16.811508   652 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.812652   652 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.756304 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:16.753852   665 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.745764   438 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:16.823799 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.822069 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:16.826188 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 152077us
I20260709 12:24:16.778472   635 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.829113 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:16.744832 32721 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.829643 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.839242 32608 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:16.839519   655 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.803399   555 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:16.822768   665 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:16.788956   507 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:16.846961   655 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.875691   665 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.894042   500 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.836189   622 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:16.913312   622 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:16.870040   555 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.913883   622 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:16.915000 31137 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:16.896807 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48615.9us
W20260709 12:24:16.935540   705 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:24:16.935341   655 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:16.940804 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:16.941422 31063 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:16.838095   350 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
W20260709 12:24:16.991266 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.916445   500 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:17.003651 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.893615 32654 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:17.022645 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 193629us
W20260709 12:24:17.036334 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 214332us
I20260709 12:24:17.040920 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1828) took 1029 ms. Trace:
I20260709 12:24:17.023391 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.044888 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:17.053324 30989 rpcz_store.cc:276] 0709 12:24:16.001658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.001829 (+   171us) service_pool.cc:224] Handling call
0709 12:24:17.030720 (+1028891us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400022bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:17.030859 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309355264}
I20260709 12:24:17.055624 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:17.057135 30989 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:24:17.058523 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48205.4us
W20260709 12:24:17.062930   711 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:24:17.059046 32608 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:17.067895 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:17.071074   684 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.892s	user 0.005s	sys 0.000s
I20260709 12:24:17.073030 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:17.073285   507 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.052861 31142 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:17.074242   438 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.094050 31694 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:17.074527   635 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:17.076557 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.077298 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:17.096002 32525 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.096342 32525 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:17.096642 32525 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.077535 31065 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:17.077941   500 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.079991 30984 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:17.080899   375 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.523s	user 0.019s	sys 0.004s
I20260709 12:24:17.082867   362 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:17.045697   591 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.012s	sys 0.013s
W20260709 12:24:17.087589 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.088059 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:17.092348 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:17.093695 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.104298 31141 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:17.089319   453 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.106405 30992 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:17.106812 30992 pending_rounds.cc:85] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 3
I20260709 12:24:17.102541 30987 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:17.097995 30986 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.010s
I20260709 12:24:17.090027 32608 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:17.108341 30986 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:17.094674   623 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.115046 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:17.104545   591 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.012s	sys 0.014s
W20260709 12:24:17.116602 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 43665.5us
I20260709 12:24:17.107111 30992 pending_rounds.cc:107] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:24:17.120975 30992 raft_consensus.cc:2889] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:17.119117 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.123248 30984 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:17.109982 32608 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.126205 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:17.126688 31137 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:17.126977 31142 raft_consensus.cc:606] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:17.102038   393 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
W20260709 12:24:17.129911   497 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:24:17.129877 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1816) took 1601 ms. Trace:
I20260709 12:24:17.130604 30992 rpcz_store.cc:276] 0709 12:24:15.528753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.528875 (+   122us) service_pool.cc:224] Handling call
0709 12:24:15.529142 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.608000 (+1078858us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5800359fd0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb7f00aa 00007f2eeb872447 00007f2eeb88afe7 00007f2eeb88c7de 00007f2eeb896b49 00007f2eeb897416 00007f2eeb893d8f 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4
0709 12:24:17.129595 (+521595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.129825 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2408747776}
I20260709 12:24:17.134011 32608 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.137013 32608 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.140591 31064 raft_consensus.cc:1217] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-6.4]   Dedup: 6.4->[]
I20260709 12:24:17.141084   638 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:17.141232 31064 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:24:17.141652   708 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.280s	user 0.004s	sys 0.000s
I20260709 12:24:17.142594 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:17.143617 31059 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:17.144251 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:17.151362   683 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.034s	user 0.011s	sys 0.005s
I20260709 12:24:17.089447 32654 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.154232 31698 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:17.131170 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.141510   516 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.152666 30889 catalog_manager.cc:5697] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:17.174439   393 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:17.174345 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:17.174932   393 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:17.175139 31142 raft_consensus.cc:606] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.175823   393 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.177484   555 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.098410   627 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.179105 31068 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:24:17.180119   627 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.097416   600 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:17.182425 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 51343.6us
W20260709 12:24:17.182636   599 consensus_meta.cc:287] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.008s
I20260709 12:24:17.183593 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.184422   584 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:17.184664 32654 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.186244 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:17.186636   595 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.514s	user 0.005s	sys 0.003s
I20260709 12:24:17.186868   597 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.187459   633 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:17.187817   633 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:17.188469   655 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.188577   600 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.125066   500 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:17.189055   655 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.189679   500 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.190500 30988 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:17.187014 31698 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:17.194202 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:17.184803   584 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:17.195917 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.215494 31065 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:17.219050 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 88398.1us
W20260709 12:24:17.197503 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 82540.5us
I20260709 12:24:17.104797   375 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.182538   591 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.141546   638 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:17.192379   500 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.193045   362 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.226348 31144 raft_consensus.cc:2376] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3. Re-sending same reply.
I20260709 12:24:17.219854 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:17.230100 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:17.223502 31068 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:17.187029   595 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:17.229666   638 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.233286 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.235374 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.226742 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.198886   600 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.266414 32525 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.195370   665 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.268885 30992 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:17.263448 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:17.246587   432 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.215245   667 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.296190 32525 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.263680 31143 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:17.270509 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.313848 30987 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:17.320182 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56803.6us
W20260709 12:24:17.321092 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 221415us
I20260709 12:24:17.315047 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:17.252429 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.258910   709 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:17.228690   516 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:17.326509 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.271274   591 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.332612 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:17.338373 31064 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:17.188649   633 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.270238   375 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.339484 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:17.339740 31698 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:17.342756 30987 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:17.347673 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 77257.5us
W20260709 12:24:17.335119   689 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:17.356338 31060 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:17.348374 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:17.356297 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:17.252169   730 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:24:17.360526 31065 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:17.361634 31061 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:17.269457   600 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.359072 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:17.336886   433 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.358491 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:17.365155 31068 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.334934   432 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:17.368650 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.370764   591 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.354930   709 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:17.378228   728 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:24:17.370538   433 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:17.379634   709 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.380128 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 311) took 1086 ms. Trace:
I20260709 12:24:17.380497   432 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.362461 32730 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.337414   423 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:24:17.387537   375 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.387280 30888 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 197) took 6159 ms. Trace:
W20260709 12:24:17.393560 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 170737us
I20260709 12:24:17.398166   627 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.355850   584 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.401484   584 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.390514 30888 rpcz_store.cc:276] 0709 12:24:11.227418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.255932 (+ 28514us) service_pool.cc:224] Handling call
0709 12:24:17.387015 (+6131083us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:14.737723 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:14.740806 (+  3083us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:14.741020 (+   214us) write_op.cc:435] Acquired schema lock
0709 12:24:14.741181 (+   161us) tablet.cc:662] Decoding operations
0709 12:24:14.896171 (+154990us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:14.896221 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:14.896262 (+    41us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:24:14.897370 (+  1108us) tablet.cc:701] Row locks acquired
0709 12:24:14.897394 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:24:14.897562 (+   168us) write_op.cc:270] Start()
0709 12:24:14.897684 (+   122us) write_op.cc:276] Timestamp: P: 1783599854897506 usec, L: 0
0709 12:24:14.897718 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:24:14.898493 (+   775us) log.cc:844] Serialized 4706 byte log entry
0709 12:24:15.651184 (+752691us) op_driver.cc:464] REPLICATION: finished
0709 12:24:15.680061 (+ 28877us) write_op.cc:301] APPLY: starting
0709 12:24:15.680131 (+    70us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:15.680362 (+   231us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:15.680444 (+    82us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:15.681771 (+  1327us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:15.681982 (+   211us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 12:24:15.682028 (+    46us) write_op.cc:312] APPLY: finished
0709 12:24:15.685011 (+  2983us) log.cc:844] Serialized 89 byte log entry
0709 12:24:15.685689 (+   678us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:15.685830 (+   141us) write_op.cc:454] Released schema lock
0709 12:24:15.685877 (+    47us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":893993,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":26721,"apply.run_cpu_time_us":2885,"apply.run_wall_time_us":6069,"num_ops":12,"prepare.queue_time_us":1365114,"prepare.run_cpu_time_us":7083,"prepare.run_wall_time_us":161312,"replication_time_us":745828,"thread_start_us":218620,"threads_started":1}]]}
I20260709 12:24:17.402438   625 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.386634   591 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.402961   625 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.403570   600 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.405658   625 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.355126   356 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
W20260709 12:24:17.406790 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.407038   356 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:17.407435   356 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:17.408314   356 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.409519   356 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.387648   723 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:17.410741   555 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:17.412410   622 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:17.414263 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45708.6us
I20260709 12:24:17.415454   665 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.415889 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.417356 31141 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:17.401898   516 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.369114   599 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
W20260709 12:24:17.388324   687 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:24:17.414393 31067 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:17.422420 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:17.388062 30889 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 2, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 2 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:17.380504 30984 rpcz_store.cc:276] 0709 12:24:16.293507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.335054 (+ 41547us) service_pool.cc:224] Handling call
0709 12:24:17.379982 (+1044928us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b540020f154. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:17.380078 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704598272}
I20260709 12:24:17.431284 31058 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:17.435155 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:17.440761 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:17.442322 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 106211us
I20260709 12:24:17.431360   555 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:17.463213 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.464449 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:17.468034 32525 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.468659 32525 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.379608   433 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.477056   638 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.447731   652 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:17.479368   652 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:17.480324   652 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.483103 31141 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:17.438769   710 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:24:17.487011 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.487602 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:17.462836   622 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.471457   600 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:17.491879 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.445825   446 raft_consensus.cc:993] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:24:17.504803 32525 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.483099   709 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:24:17.451478 32721 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:17.507143   555 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.438412   423 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 12:24:17.515961 32727 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.777s	user 0.015s	sys 0.005s
I20260709 12:24:17.516389 32727 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.490010 32645 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.509248   622 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.489989 32654 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:24:17.519222   622 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.514892   446 raft_consensus.cc:1081] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:24:17.519985   600 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:17.512575 31068 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
W20260709 12:24:17.512585 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49489.1us
I20260709 12:24:17.521126   600 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.521611   622 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.522063 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.514802 32721 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:17.522653   735 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:24:17.530884 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.531752 31698 raft_consensus.cc:2368] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:17.533207   604 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:17.533210 32645 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:17.533818   604 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:17.534319 31692 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:24:17.534698 31692 pending_rounds.cc:85] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 4
I20260709 12:24:17.534482 32645 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.535112 31692 pending_rounds.cc:107] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:24:17.534715   604 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.535413 31692 raft_consensus.cc:2889] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.536051   604 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.538679 31143 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:17.539098 31143 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:17.539870 31143 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.514187   638 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:17.540660 30988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1820) took 1987 ms (client timeout 2060 ms). Trace:
W20260709 12:24:17.541043 30988 rpcz_store.cc:269] 0709 12:24:15.553253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.892351 (+339098us) service_pool.cc:224] Handling call
0709 12:24:17.540503 (+1648152us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400225e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:17.540600 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2982467840}
I20260709 12:24:17.543249 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.526794 32721 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.547992 31695 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:17.526938   375 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:17.542024   712 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:17.551360   712 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:17.552284   712 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.553828   712 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:17.546957 32727 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.549579   579 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:24:17.555217   579 consensus_queue.cc:1243] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:17.550813 32654 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:17.556015 32654 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:17.556865 32654 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.557950 31696 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:17.558776 31694 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:17.559103 32654 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.559207   393 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.559798 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.560770 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.561358 31689 raft_consensus.cc:2376] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6. Re-sending same reply.
I20260709 12:24:17.527724   635 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:24:17.569671   655 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.570549 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:17.571218   633 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.571573 32554 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.547997 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.571954 32554 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:17.572335 32554 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.575827 31143 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.576251 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:17.576874 30988 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:17.580168 32721 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.582387 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.583498 31696 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:17.584046   667 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:17.585062 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.585858 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.586935 31689 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:17.587535 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.587671 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:17.589479 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.591254   555 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:17.591830   439 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:24:17.593019 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.598981 31698 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:17.672654 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:17.666942 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:17.601320 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.602748 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:17.608219 30887 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 312) took 9094 ms. Trace:
I20260709 12:24:17.609524 31143 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:17.638880 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:17.662577 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:17.676733 31062 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:17.664578 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 50626.8us
I20260709 12:24:17.674468 31064 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:17.640151 32554 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.679020 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:17.679085 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:17.667454 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:17.680172 31060 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:17.601120 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.680312 30987 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:17.684438 30955 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:17.686260 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:17.686597 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.687641 30991 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:17.688416 31140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:17.689116 31140 raft_consensus.cc:606] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:17.686753 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:17.704710 30986 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:17.707996 32554 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:17.736099 30984 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:17.742079 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.738422 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.692243 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:17.743777 31059 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:17.744035 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.745105 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.748066 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:17.751580 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 383761us
I20260709 12:24:17.700608   393 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.752694 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.760912 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:17.765204 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:17.767388 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.753368 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:17.756158   393 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.771309 31697 raft_consensus.cc:2368] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:17.776378 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:17.768672 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:17.754768   730 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.729743   423 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:24:17.774070   433 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:24:17.799556 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.790508   736 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:17.801944 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.772400   638 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:17.769460 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.754456   579 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:24:17.824033 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.824085 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.824815 31697 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:17.795975   393 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.794099   730 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:17.824160 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.803638   710 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:17.800993   622 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.829041 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.829638 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.805068   433 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:17.833269 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:17.838310 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.823683   667 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:17.825841   597 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:17.675981 30887 rpcz_store.cc:276] 0709 12:24:08.513553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.513795 (+   242us) service_pool.cc:224] Handling call
0709 12:24:08.614026 (+100231us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fe090 00007f2eeb89cf3a 00007f2ef1652e9c 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:17.608011 (+8993985us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:13.604987 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:13.747456 (+142469us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:13.747506 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:24:13.747534 (+    28us) tablet.cc:662] Decoding operations
0709 12:24:13.802759 (+ 55225us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:13.829127 (+ 26368us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:13.853587 (+ 24460us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:24:13.888924 (+ 35337us) tablet.cc:701] Row locks acquired
0709 12:24:13.891460 (+  2536us) write_op.cc:260] PREPARE: finished
0709 12:24:13.938891 (+ 47431us) write_op.cc:270] Start()
0709 12:24:13.965386 (+ 26495us) write_op.cc:276] Timestamp: P: 1783599853911201 usec, L: 0
0709 12:24:13.972699 (+  7313us) op_driver.cc:348] REPLICATION: starting
0709 12:24:14.272983 (+300284us) log.cc:844] Serialized 1308 byte log entry
0709 12:24:15.488618 (+1215635us) op_driver.cc:464] REPLICATION: finished
0709 12:24:15.665792 (+177174us) write_op.cc:301] APPLY: starting
0709 12:24:15.670516 (+  4724us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:15.676854 (+  6338us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:15.677007 (+   153us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:15.677521 (+   514us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:15.677704 (+   183us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:24:15.677752 (+    48us) write_op.cc:312] APPLY: finished
0709 12:24:15.677951 (+   199us) log.cc:844] Serialized 35 byte log entry
0709 12:24:15.679106 (+  1155us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:15.679211 (+   105us) write_op.cc:454] Released schema lock
0709 12:24:15.679254 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:15.679657 (+   403us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:15.679690 (+    33us) write_op.cc:454] Released schema lock
0709 12:24:15.898969 (+219279us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:15.899051 (+    82us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2475917,"reported_tablets":16,"spinlock_wait_cycles":35151104,"child_traces":[["op",{"apply.queue_time_us":163402,"apply.run_cpu_time_us":2214,"apply.run_wall_time_us":18275,"num_ops":3,"prepare.queue_time_us":1094310,"prepare.run_cpu_time_us":32155,"prepare.run_wall_time_us":1168403,"raft.queue_time_us":938421,"raft.run_cpu_time_us":31407,"raft.run_wall_time_us":420322,"replication_time_us":1487548,"thread_start_us":1524842,"threads_started":4,"wal-append.queue_time_us":599771,"wal-append.run_cpu_time_us":18510,"wal-append.run_wall_time_us":1809579}]]}
I20260709 12:24:17.846195 32554 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.847085 32554 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.829638   730 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.848933 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.849932 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:17.850118 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:17.850796 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.805783   665 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:24:17.833438   710 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:17.832628   433 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:17.830603 31690 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:17.862280 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.839279   736 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:17.838595 31143 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:17.864311 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:17.829365   591 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:24:17.875352 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:17.867970 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.871104 32554 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:17.873229   554 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:24:17.843156   638 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:17.870431   375 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:24:17.879498   729 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.852548 32721 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:17.846603   597 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:17.865269   625 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.836843   622 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.888244 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.892091   738 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:17.852855   579 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.863405   710 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:17.859747   727 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:17.888291   743 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:24:17.913462   723 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.930871   723 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.931231   723 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.921093 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:17.926362   745 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:24:17.926926   579 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.929363   597 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.936718   597 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.896745 30988 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:17.944980 31068 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:17.945940 31059 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:24:17.946579 31059 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:17.946960 31059 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.897648 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.900605   433 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.865729 32730 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:24:17.960093 31061 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:17.911154 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:17.912039   591 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:17.959125   433 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.961421 31144 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:17.975715 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:17.977411 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:17.979223 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:17.989739 31690 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:17.969877   591 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:17.990928   591 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:17.992143   591 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.993338 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:17.994424   739 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.994926   739 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:17.997018   739 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:18.001744 30983 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:18.003921 31695 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:18.008909 31694 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:18.012270 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:17.970721   729 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:17.970965 31689 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:18.024798 31698 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:18.026878 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.027699 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:18.036099 31696 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:18.006271   579 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:17.980196   667 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:24:18.006799   597 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:17.969638   638 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.021347 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:18.024062   751 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:18.057149 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 192862us
I20260709 12:24:18.048161 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:18.020053 31059 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.043565 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:18.072075 31139 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:17.421376   507 consensus_queue.cc:1175] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" responder_term: 4 status { last_received { term: 3 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:18.080482 31664 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:18.085601 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:18.087057 31060 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.043534   738 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:24:18.087868 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:18.088296 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:18.089329 30992 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:18.061229   597 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.092989 31139 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.095206 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:18.093961   500 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:18.064945   751 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.095674   722 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:18.109069 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:18.109669 31060 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:18.112509 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:18.113863 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:18.100163   599 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.097973   729 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.149962 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:18.151499 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:18.154249 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:18.140302   722 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:18.141770   500 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:18.142652   432 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.221474   432 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.221951   500 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.224135   432 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:18.162708 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48936.1us
W20260709 12:24:18.157299 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45874.5us
I20260709 12:24:18.155699 31063 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:24:18.212119   760 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:18.151275   739 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:18.137863 32721 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:24:18.143797 32645 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:18.225318 31063 pending_rounds.cc:85] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:24:18.237167 31063 pending_rounds.cc:107] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:24:18.140975   599 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.212951   738 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:18.153076 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:18.153867   555 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:18.237435 31063 raft_consensus.cc:2889] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:18.150875   736 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:18.196353 32608 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:18.256235 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 100391us
I20260709 12:24:18.228245   760 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:18.260609   710 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:18.255492   750 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:24:18.269048 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:18.242446   757 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:18.244423 32645 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.262360 31059 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:18.240567   739 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:18.292596 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.288586 31142 raft_consensus.cc:606] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:18.279757   762 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:18.278656 32608 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:24:18.256692   736 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.234917   730 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.289880   597 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:18.333334 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 102058us
I20260709 12:24:18.302678   729 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:18.305472   751 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:18.297091   710 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:18.341656 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:18.313630 32608 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:18.292605   722 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.318562   736 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:18.319439 32645 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:18.323642 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:18.324210   726 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:18.343456 31140 raft_consensus.cc:2410] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:18.343562 30989 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 2: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:18.325150 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:18.327205   622 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:18.327576 31693 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
W20260709 12:24:18.308286   753 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:24:18.337541 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:18.378407 31144 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:18.378815 31144 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:18.340179 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:18.310849   766 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:18.354775 30986 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:18.334295   729 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:18.304836   393 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.374601   599 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:18.343672 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:18.401813 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58265.2us
I20260709 12:24:18.376413 30988 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
W20260709 12:24:18.343958   754 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:18.399576 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:18.342971 32608 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:18.404224 31142 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:18.369732   726 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:18.370370   751 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:18.403954 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 175425us
I20260709 12:24:18.395377   497 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:18.406179 31144 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.376341   600 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:18.409092 30987 raft_consensus.cc:2368] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
I20260709 12:24:18.399786 32645 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.416581 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:18.406667   710 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.421146   762 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:18.421460   433 raft_consensus.cc:993] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:24:18.302798   760 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.421566   762 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.422029   433 raft_consensus.cc:1081] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Signalling peer e9cccf7eb2a545ef8c2169b67ff261bf to start an election
I20260709 12:24:18.422087   762 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.422513   729 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.415256   739 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:18.428124   600 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.428390   723 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.418650 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:18.438942 32608 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.410022 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:18.442999 31059 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:18.444085   768 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:18.444180 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:18.444674   768 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.445261   597 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:18.445717   597 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:18.446072   597 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:18.420612 32645 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:18.449199   625 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
I20260709 12:24:18.415949   751 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:18.450389 31137 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
W20260709 12:24:18.450852 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.430555   497 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.420830   710 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.430743   599 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:18.429888   762 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.457271   625 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:18.588515   757 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:18.628753   591 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:18.450899   751 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.634966   599 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.594278   710 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.660298   730 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:18.662721 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.676944 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.663329 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.663655 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 256280us
W20260709 12:24:18.665073 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.667383 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:18.667814 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 410205us
I20260709 12:24:18.667919 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:18.671474 31061 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:18.678380 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:18.672581 31062 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:18.684818 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.667385   393 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:18.687608 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:18.667745   736 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:18.673017 32608 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.671846   432 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:18.692065 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:18.673512   591 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.703748   432 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.704546   736 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.683354 32730 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:18.698655   480 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.694095 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:18.671597   757 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.691349   600 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:18.689163   730 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:18.765692 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:18.695032   723 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:18.766919 31067 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.735641 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 428) took 1199 ms. Trace:
W20260709 12:24:18.765329 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 71650.9us
I20260709 12:24:18.773911 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:18.732795 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:18.769361 31068 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:18.767026 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 59619.4us
I20260709 12:24:18.730189 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:18.695572   638 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:18.708309 32645 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:18.775030 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:18.706688 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:18.773896   375 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.775756 31063 rpcz_store.cc:276] 0709 12:24:17.536156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.078455 (+542299us) service_pool.cc:224] Handling call
0709 12:24:18.091709 (+ 13254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.255224 (+163515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.735368 (+480144us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b54002832d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:18.735570 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115867904}
I20260709 12:24:18.777050 31062 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.778654 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:18.780736 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:18.721082 31691 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:18.788699 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:18.789523 31068 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:18.768860   730 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:18.780575 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47889.2us
I20260709 12:24:18.721694   497 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:18.717607   591 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:18.786072 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:18.796553 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:18.780390 31058 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:18.784821 31061 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:18.716542   432 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:18.708731   722 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
W20260709 12:24:18.798842 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 131535us
I20260709 12:24:18.793022 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 691) took 1204 ms. Trace:
I20260709 12:24:18.729647   597 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.810367 31691 rpcz_store.cc:276] 0709 12:24:17.588847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.686274 (+ 97427us) service_pool.cc:224] Handling call
0709 12:24:18.720898 (+1034624us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54002b19a0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb80bdc0 00007f2eeb897e18 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:18.792925 (+ 72027us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":899799296}
I20260709 12:24:18.811141 31060 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:18.783979   604 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:18.801290 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 111847us
I20260709 12:24:18.779561   638 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.804734 32654 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.807013 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 675) took 1250 ms. Trace:
I20260709 12:24:18.786463   423 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:18.809062   735 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:18.809566 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:18.769862   480 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.815742 31063 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:18.792498 32645 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.787678   375 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.816584 31693 rpcz_store.cc:276] 0709 12:24:17.556977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.559543 (+  2566us) service_pool.cc:224] Handling call
0709 12:24:18.806831 (+1247288us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400511b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:18.806952 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894825856}
I20260709 12:24:18.824141 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:18.824373   375 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:18.820283 31067 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:18.798475   730 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.802049   606 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.816550   638 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.801645   579 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:18.831121 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:18.812741   772 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:24:18.842151   480 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:18.838310 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:18.843655   741 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.966s	user 0.002s	sys 0.012s
W20260709 12:24:18.827580   776 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:24:18.846918   579 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:18.832803   423 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:18.849853 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:18.825889   599 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:18.865351 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68921us
I20260709 12:24:18.865994 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:18.863155 31062 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:18.856113   600 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:18.856308   579 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:18.874915   600 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.876436 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:18.877743 31062 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:18.885042   604 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:18.885589   579 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.885079   423 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.888576   638 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:18.890031   749 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.016s	user 0.006s	sys 0.014s
I20260709 12:24:18.875919   762 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:18.891788 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:18.885599   604 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:18.899052   772 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:18.898921   604 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.900262   604 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.900902   710 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.903009   710 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:18.904989 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:18.906049 31106 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:18.916764   732 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.917335   732 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:18.918183 32654 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:18.918602 32654 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.918932 32654 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.919471   732 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:18.920331 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:18.921655 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:18.947366 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.948601   783 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:18.949223   783 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:18.949607   783 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.948516 31137 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:18.948737 31138 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
W20260709 12:24:18.930469   665 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.023s	user 0.010s	sys 0.025s
I20260709 12:24:18.958988   735 raft_consensus.cc:993] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:24:18.963493   735 raft_consensus.cc:1081] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
W20260709 12:24:18.964435 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 98480.4us
W20260709 12:24:18.952863   746 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.066s	user 0.011s	sys 0.018s
W20260709 12:24:18.967543   746 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.011s	sys 0.019s
I20260709 12:24:18.961220   579 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.969895 31061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:24:18.970477 31061 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:18.970798 31061 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.959916   737 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:18.966365 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1860) took 1355 ms. Trace:
I20260709 12:24:18.966519 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:18.979321 31062 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
W20260709 12:24:18.966388 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.962292   665 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.011s	sys 0.025s
I20260709 12:24:18.991421 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:18.977471 30985 rpcz_store.cc:276] 0709 12:24:17.611397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.611534 (+   137us) service_pool.cc:224] Handling call
0709 12:24:17.672058 (+ 60524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.700087 (+ 28029us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:24:17.700126 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.739743 (+ 39617us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.848660 (+108917us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:17.848707 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.848768 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.889492 (+1040724us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:18.960366 (+ 70874us) raft_consensus.cc:1685] finished
0709 12:24:18.960390 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.966302 (+  5912us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5231,"raft.queue_time_us":16484,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13087,"spinlock_wait_cycles":92904704,"thread_start_us":77127,"threads_started":1,"wal-append.queue_time_us":114115}
I20260709 12:24:18.989110 31061 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:18.951893   710 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:24:19.001972 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:19.007050 31061 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:19.012243   748 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.118s	user 0.000s	sys 0.006s
I20260709 12:24:19.012564   767 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.013250   767 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.015965   767 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.016325   757 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.018960 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1465) took 1333 ms. Trace:
I20260709 12:24:19.002406   600 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.023625   783 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.024259   726 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.022650 31136 rpcz_store.cc:276] 0709 12:24:17.685984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.688686 (+  2702us) service_pool.cc:224] Handling call
0709 12:24:17.701316 (+ 12630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.701830 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.701857 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.740477 (+ 38620us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.823286 (+ 82809us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:17.823341 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.823420 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.299323 (+475903us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b4800251958. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef07bdaa1 00007f2ef07b4b37 00007f2ef07b49ca 00007f2ef08ac26f 00007f2ef08abf37 00007f2ef08c055c 00007f2ef08c0508 00007f2ef08c0488 00007f2ef08c0443 00007f2ef08bf6ec 00007f2eef7e9a96
0709 12:24:19.011552 (+712229us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:19.018757 (+  7205us) raft_consensus.cc:1685] finished
0709 12:24:19.018778 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.018905 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8377,"raft.queue_time_us":583,"raft.run_cpu_time_us":4606,"raft.run_wall_time_us":598343,"spinlock_wait_cycles":35340800,"thread_start_us":58341,"threads_started":1,"wal-append.queue_time_us":87494}
I20260709 12:24:19.025441   726 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:19.027590   783 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:19.040052   757 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:19.046718 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:18.997994   604 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:24:19.000349   761 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:24:19.077913 31058 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:19.047863 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 56540.1us
I20260709 12:24:19.106606 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:19.036275   606 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:19.112560   791 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.051s	user 0.005s	sys 0.000s
I20260709 12:24:19.118290 31109 leader_election.cc:400] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:19.122701 32654 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.040714   638 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.126109 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:19.127223 30955 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:19.127923 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1867) took 1288 ms (client timeout 1558 ms). Trace:
I20260709 12:24:19.128072   497 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:19.128589   497 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:19.129813 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:19.129629   497 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.116195   710 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:19.131253   710 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.131587   710 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.132558 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:19.133661 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:19.134371   606 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:19.135251   606 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:19.116588 32721 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:19.045701   757 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.136091 32721 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:19.128319 30987 rpcz_store.cc:269] 0709 12:24:17.839727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.839840 (+   113us) service_pool.cc:224] Handling call
0709 12:24:19.127671 (+1287831us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400225e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:19.127854 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1281592192}
I20260709 12:24:19.139811 31061 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:19.131644 31058 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:19.127238 31144 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:19.137305 31065 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:24:19.142506 31065 pending_rounds.cc:85] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 2
I20260709 12:24:19.142719 31065 pending_rounds.cc:107] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:24:19.142963 31065 raft_consensus.cc:2889] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:19.144138 31106 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:19.144845 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.145802 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.146008 31141 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:19.143844 32608 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.045675   480 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:19.085942   769 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:19.146498 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 292) took 1124 ms. Trace:
I20260709 12:24:19.146940   765 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:19.128198   600 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:19.138140 32721 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.143985   599 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.147979   751 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.156697 31141 rpcz_store.cc:276] 0709 12:24:18.022519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.024773 (+  2254us) service_pool.cc:224] Handling call
0709 12:24:19.129643 (+1104870us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:19.146448 (+ 16805us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5462272}
I20260709 12:24:19.150701   480 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.163004 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:19.151397   604 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:19.158888 32654 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:19.147315 32608 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.154114   730 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.179879 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:19.179988   730 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.165776   760 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:19.155692   769 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.182814   768 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.183984   736 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.184363   769 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.167331   600 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:19.186899   600 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.162968   599 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.167784   751 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.169814   627 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:19.137434   757 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.170516   480 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.161335 31144 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:19.179988 32608 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.191812 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:19.194767   730 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:19.196741   779 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:19.219656 31064 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:19.219384   779 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.202636   393 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:19.221690   393 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.222121   779 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.223001   599 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.200913 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:19.201778 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:19.174065   453 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:19.203399 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.211043 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:19.173707 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:19.198825   627 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:19.223697   393 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.210107 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:19.194487   604 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:19.215656 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:19.217731   751 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.218283   730 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.281474   769 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:19.206367 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:19.289948 31694 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:19.290740 31065 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:19.212992 31698 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
W20260709 12:24:19.292965 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.296281 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 86240.8us
W20260709 12:24:19.300709 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 123710us
I20260709 12:24:19.303395 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:19.285301 31691 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:19.306982 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.307351   433 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:19.307813   599 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:19.307778   433 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:19.312027   433 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.312704 30992 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:19.313690 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:19.308671 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 106998us
I20260709 12:24:19.311211 31694 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:19.290261   453 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:19.316346 30984 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:19.314960   779 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:19.317378   599 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.303990   769 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.303946   606 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:19.322085   730 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.300587   736 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:19.340626 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.340969   736 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:19.341892   736 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.342968 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.263331   722 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:24:19.343001 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.344233   722 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:19.323695 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:19.345155 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:19.345800 31693 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:19.327549 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:19.329582 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:19.330340 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:19.348729 31141 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:19.349035 31141 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:19.309001   393 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.323696   597 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.349648 31141 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.254455   710 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.356226   393 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.318938   604 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.339175   606 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:19.358165   393 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.358204   606 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:19.336160   579 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.362545   579 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.362840   480 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.363920 31139 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:19.358283 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 287) took 1379 ms. Trace:
I20260709 12:24:19.365298   769 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.365057 31140 rpcz_store.cc:276] 0709 12:24:17.978747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.978905 (+   158us) service_pool.cc:224] Handling call
0709 12:24:19.358080 (+1379175us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540029efd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:19.358211 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835318528}
I20260709 12:24:19.367409   599 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.369479 31144 raft_consensus.cc:2368] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4.
I20260709 12:24:19.337316 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:19.337589 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.370040 31141 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.339564 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.344638   722 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:19.352506 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:19.352653 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:19.353904 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:19.335731   453 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:19.379983 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:19.400104 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:19.377429 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:19.381197 31141 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:19.395495 30988 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:19.382912 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.398525 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.407898 30989 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:19.408476 30992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:49522
I20260709 12:24:19.408999 30992 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:19.409389 30992 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.381476   709 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:19.419221 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.381281 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:19.396607 32654 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:19.429260 31109 leader_election.cc:400] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
I20260709 12:24:19.429669 31109 leader_election.cc:403] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:19.409089 32721 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:24:19.431692   782 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.405562 32608 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:19.432394 32608 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:19.433228 32608 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:19.434046   788 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.368s	user 0.000s	sys 0.013s
I20260709 12:24:19.436590   709 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.439209   710 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:19.439465 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:19.439857   579 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:19.427654   446 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:19.441461   709 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.403590 31691 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:19.398195 31138 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.394733   772 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:19.405683   769 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:19.446758 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.450169 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.450927 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.451782 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.465337 30992 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.418663   760 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:19.417605 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:19.478188 31061 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
W20260709 12:24:19.478487 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 60964.9us
I20260709 12:24:19.479070 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.479732 31061 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:19.480580 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.480688 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:19.481185 31063 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.481556   751 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:19.375344   497 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:24:19.476553 30983 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:19.482086   751 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:19.483043   751 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.483443 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.484680 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:19.485215 31062 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.486673 31067 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:19.490458 30986 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:19.490960   774 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.491392 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:19.491463   774 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.492045 30988 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.493530   774 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:19.496383 31138 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:19.497999 30992 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.486626 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.505273 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:19.463052 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.507877 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:19.509838 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.510141 30990 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:19.501797 30987 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:19.517767 31141 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:19.519239 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.520028 31141 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:19.521404 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:19.441638   433 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.513545 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.481144 31061 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:19.530948 30988 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:19.512338 30992 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:19.419056   729 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.529791 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.532817 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.532984 31063 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:19.533748 31140 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:19.474625   597 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:19.477849   769 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.478128   760 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.534190   761 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:19.527935   591 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:19.541026 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.475135   737 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 12:24:19.542642 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.454427 32654 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.498327 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.544541 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:19.622917 30986 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.535308 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:19.624043 30989 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.535197 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.545109 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:19.625270 30983 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
W20260709 12:24:19.625399 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 80369.6us
I20260709 12:24:19.549594 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.534545 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:19.534955   709 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.541882   606 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.571993 31136 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:19.593381 31062 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:19.570937   760 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.596972 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.576733 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:19.599174 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 45818.9us
I20260709 12:24:19.635151 30992 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.599849 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:19.635814 31058 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.544006   726 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:19.534729 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:19.603843   599 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.551235   769 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:19.498399   722 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.629626 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:19.665102 31142 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:19.596230 31141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:19.665889 31141 raft_consensus.cc:606] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:19.666239 31141 raft_consensus.cc:614] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:19.667443 31141 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:19.633491 31139 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:19.571800   766 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:19.571226   712 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:19.603231   393 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:19.620520   633 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:19.633478   768 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.630390   423 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:19.682535 31139 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:19.699961 30992 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:19.660059   736 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.622397   446 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 12:24:19.688936 31141 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:19.573418   597 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.627539 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.650564   726 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.602626   761 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.692111 30989 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:19.632702   709 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.685880 31058 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:19.752081 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 348193us
I20260709 12:24:19.700982 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:24:19.686874 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 236951us
I20260709 12:24:19.688967   710 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:19.695047   749 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:24:19.763239 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 356874us
I20260709 12:24:19.632263   591 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.683389 31143 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:19.657744 30986 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:19.773324 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:19.758072   722 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.623162   627 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.777024 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:19.774984 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:19.689749   599 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:19.775399 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 286352us
I20260709 12:24:19.761770   423 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:19.813786 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:19.692700   712 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.768105   726 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:19.753079   768 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.701145   633 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:19.892385 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 115420us
I20260709 12:24:19.695648   393 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.695469   766 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.762863   736 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:19.812448   710 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:19.915838   751 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:19.901702   815 raft_consensus.cc:2764] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:19.920228   815 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:19.920666   815 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:19.921730   710 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:19.921548   815 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:20.040190 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:24:19.921494   749 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:24:19.899278   627 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:19.929358   393 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:20.110735 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 447935us
I20260709 12:24:19.923122   606 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:19.912948   782 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
I20260709 12:24:19.927742   633 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:20.088428   433 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:20.099475 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54372us
W20260709 12:24:20.102587 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.901366   712 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:19.930745   597 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.904664   591 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.927579   709 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:20.133997 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:19.899096   752 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:19.929080   599 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.926167   766 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.930830   768 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.907612   761 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:19.902880   814 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.188s
W20260709 12:24:20.228315 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 453381us
I20260709 12:24:20.177587 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:20.234043 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 56568us
I20260709 12:24:20.240547 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:20.162501 31144 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:20.131403   633 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:20.270833 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 491818us
I20260709 12:24:20.272262 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:20.221820   709 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.316848 31060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.005s	sys 0.052s
W20260709 12:24:20.318480 31060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.005s	sys 0.052s
W20260709 12:24:20.319300 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47119.7us
W20260709 12:24:20.238400   809 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:20.229763 32654 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:20.229437   599 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.228173   765 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:20.232537   766 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:20.271123   433 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:20.352772 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:20.316741   768 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.385500 31058 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.131649   423 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:20.338327   735 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.253s	user 0.020s	sys 0.036s
W20260709 12:24:20.355423 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 96090.8us
I20260709 12:24:20.231580   726 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:20.289747   606 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 12:24:20.318653   633 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:20.385949   736 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:20.550007 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.772401 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 531923us
I20260709 12:24:20.825730   633 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:20.799330   772 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.826546   633 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.826629   735 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.746s	user 0.033s	sys 0.062s
I20260709 12:24:20.826712   772 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:20.498909 31030 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.010s	sys 0.035s
I20260709 12:24:20.722203   761 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.828081 31030 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.021s	sys 0.039s
I20260709 12:24:20.828414   761 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:20.495361 30990 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.003s	sys 0.009s
W20260709 12:24:20.830061 30990 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.014s	sys 0.013s
I20260709 12:24:20.830317   829 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:20.406934   752 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:24:20.724143   816 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 12:24:20.831722   752 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:20.832886   768 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:20.833232   736 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.833962   736 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:20.367236   599 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:20.836580   736 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:20.797923   834 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 12:24:20.421379   722 consensus_meta.cc:287] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.004s
I20260709 12:24:20.830817   829 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:20.862448 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:20.842973   761 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:20.844491   840 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.369951   807 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
W20260709 12:24:20.845450 31698 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.006s	sys 0.009s
W20260709 12:24:20.845817 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.017s	user 0.000s	sys 0.015s
W20260709 12:24:20.846081 30984 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.039s	user 0.002s	sys 0.013s
W20260709 12:24:20.464941 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 78641.2us
I20260709 12:24:20.847046 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:20.847596 31058 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:20.353925   393 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:20.421031   453 consensus_meta.cc:287] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.015s	user 0.008s	sys 0.001s
W20260709 12:24:20.859287 30986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.007s	sys 0.004s
I20260709 12:24:20.827162   633 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:20.837190   599 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:20.829519   772 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:20.865010 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1826) took 1523 ms. Trace:
I20260709 12:24:20.389881   766 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:20.857342   729 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:20.889209 31698 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.006s	sys 0.009s
W20260709 12:24:20.889451 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.000s	sys 0.015s
W20260709 12:24:20.971470 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 1.15769e+06us
I20260709 12:24:20.837949   827 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:20.845469   726 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:20.820330   830 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:20.334316   709 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:20.946030   760 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.285s	user 0.006s	sys 0.001s
I20260709 12:24:20.921501 31142 rpcz_store.cc:276] 0709 12:24:19.341187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.342355 (+  1168us) service_pool.cc:224] Handling call
0709 12:24:19.342674 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.665533 (+322859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.864796 (+1199263us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54001c61d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:20.864953 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741034752}
W20260709 12:24:20.982182 30984 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.003s	sys 0.013s
I20260709 12:24:20.935985   633 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:20.904989 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:21.026152 31697 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:20.922302 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.027287 31058 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:20.924214   766 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:20.820822   828 raft_consensus.cc:2764] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:24:20.995196 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 132783us
I20260709 12:24:20.997251   825 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:21.019776   712 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:20.900831 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:20.946667   729 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:20.917804   840 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.022892   599 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:20.999907 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 109933us
I20260709 12:24:20.904991   393 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:20.972222 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:20.938336 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:21.031265 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 184280us
I20260709 12:24:20.990607   433 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:20.992017   752 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:21.034464   822 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:20.904577 30986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.008s	sys 0.004s
I20260709 12:24:20.921973   453 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.037914 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:20.385212   765 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.038937 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.054579 31058 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:21.039410   841 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.017795   736 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:21.041378 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 140623us
I20260709 12:24:21.062937 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:21.023039 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 336) took 1691 ms. Trace:
I20260709 12:24:21.044162 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:21.033989   835 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.990845   761 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.050685 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:21.055739 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:20.906872 32654 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.125s
I20260709 12:24:21.102166   845 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.102698   845 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.103142   845 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:21.104138 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60068us
I20260709 12:24:21.104884 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:21.105134 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1853) took 1649 ms. Trace:
I20260709 12:24:21.105890 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:21.105517 31067 rpcz_store.cc:276] 0709 12:24:19.456013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.489034 (+ 33021us) service_pool.cc:224] Handling call
0709 12:24:19.489300 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.771683 (+282383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.771713 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.771753 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:19.771783 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.104923 (+1333140us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54001bb054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:21.104966 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.105080 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648983296}
I20260709 12:24:21.107352 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.108117 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.108938 31067 raft_consensus.cc:2410] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:21.110773 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1854) took 1654 ms. Trace:
I20260709 12:24:21.111114 31062 rpcz_store.cc:276] 0709 12:24:19.456295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.648591 (+192296us) service_pool.cc:224] Handling call
0709 12:24:19.693497 (+ 44906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.806349 (+112852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.806388 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.806432 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:19.806462 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.110568 (+1304106us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54002ce7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:21.110608 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.110721 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259059072}
I20260709 12:24:21.102169   846 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.054463 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 703) took 1742 ms. Trace:
I20260709 12:24:21.049765   766 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.059214 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.125415 30985 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:21.041077   825 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:21.116369 31698 rpcz_store.cc:276] 0709 12:24:19.311656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.311773 (+   117us) service_pool.cc:224] Handling call
0709 12:24:19.312067 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.312629 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.312655 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:19.335073 (+ 22418us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.779327 (+444254us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:20.970464 (+1191137us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.053862 (+ 83398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.054276 (+   414us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.054314 (+    38us) raft_consensus.cc:1685] finished
0709 12:24:21.054325 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.054417 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49556,"prepare.queue_time_us":695347,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":224824,"raft.queue_time_us":63271,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":12892,"thread_start_us":1589824,"threads_started":3,"wal-append.queue_time_us":736801}
I20260709 12:24:21.044821   735 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.045032   772 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:21.032541   760 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.073922 31058 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:21.074193 30984 rpcz_store.cc:276] 0709 12:24:19.331636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.332200 (+   564us) service_pool.cc:224] Handling call
0709 12:24:19.332516 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.332762 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:19.349850 (+ 17088us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.022620 (+1672770us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.022689 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:21.022752 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.022838 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.022980 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16949,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":306724,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":4067,"spinlock_wait_cycles":69237632,"thread_start_us":1672346,"threads_started":2,"wal-append.queue_time_us":736354}
I20260709 12:24:21.076637   433 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.077201   840 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:21.077728   843 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.077554   841 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.079313   729 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:21.076665 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.081534 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1849) took 1996 ms. Trace:
I20260709 12:24:21.149174 31060 rpcz_store.cc:276] 0709 12:24:19.085381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.104924 (+ 19543us) service_pool.cc:224] Handling call
0709 12:24:19.105298 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.110952 (+  5654us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:19.355925 (+244973us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b18000f1de0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf 00007f2eeb8ae038
0709 12:24:19.686678 (+330753us) log.cc:844] Serialized 14 byte log entry
0709 12:24:20.319521 (+632843us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.337966 (+ 18445us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.338125 (+   159us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:20.338171 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.338254 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.081230 (+742976us) raft_consensus.cc:1685] finished
0709 12:24:21.081337 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.081481 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5133,"prepare.queue_time_us":90207,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":8021,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":3657,"spinlock_wait_cycles":9173888,"thread_start_us":875316,"threads_started":2,"wal-append.queue_time_us":1222909}
I20260709 12:24:21.083879   761 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:21.152361   627 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.041s	user 0.008s	sys 0.000s
I20260709 12:24:21.152418   761 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:21.049549   712 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:21.086794 30992 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:21.132133   772 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:21.087406   769 consensus_meta.cc:287] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.393s	user 0.009s	sys 0.002s
I20260709 12:24:21.088846   829 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.089293   393 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.053503 31059 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:21.063115   768 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.091447   453 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.091804   736 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.092660   709 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.061216   838 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.098351   835 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.049783   446 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.174522   446 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.176542   446 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.061787   818 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:21.099911 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:21.177809 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 77974.9us
I20260709 12:24:21.178485 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:21.179337   736 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.179889   722 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.181051 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.181634 31061 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:21.051513   599 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.182708 32721 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.100960 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:21.101567 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 50957.3us
I20260709 12:24:21.183537 30987 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:21.184649 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.127900 32654 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:21.011230   827 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.130750   853 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.102041   842 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.145205   851 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.145546 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:21.133471   735 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.146322   843 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.146947   760 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.146631   825 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.148231   765 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:21.148415   841 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.153249 31065 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:24:21.056051   822 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.153081   627 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.155628 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.228204 31064 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:21.155826 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 183669us
I20260709 12:24:21.160725 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.228814 31067 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:21.228979 31033 leader_election.cc:400] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4.
I20260709 12:24:21.229274 31033 leader_election.cc:403] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:21.157119   830 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:21.232353 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.161672   847 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:21.165356 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:21.153657   772 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:21.153600   712 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:21.171408   803 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.602s	user 0.024s	sys 0.062s
I20260709 12:24:21.156200   769 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.172381   838 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.166489   393 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.131505   766 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.186239 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:21.185760 32654 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.188505 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.194427 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.197746 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.198320 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.200330 30987 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:21.190054 32721 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:24:21.212186 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:21.212983 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 81911.5us
I20260709 12:24:21.221871 31065 pending_rounds.cc:85] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:24:21.188433   853 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.229988 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:21.229663 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.232173 31033 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:21.174031   845 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.206689   851 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.210692   709 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.200567   842 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.232934 31064 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:21.504436 32645 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:21.505257 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.505172 32645 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.505954 31064 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:21.506587 30988 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 1.
I20260709 12:24:21.507606 32645 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.229806   855 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:21.507982 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:21.508399   855 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.344115   765 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:21.503913   827 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:21.504184   802 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.935s	user 0.009s	sys 0.018s
W20260709 12:24:21.491586   837 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.455s	user 0.000s	sys 0.000s
I20260709 12:24:21.504709 32654 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.496248 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.519877 30985 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:21.520606 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:21.521085 30985 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:21.506193 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.522151 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.456266   762 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:21.510845 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 339) took 1935 ms. Trace:
I20260709 12:24:21.523058 30990 rpcz_store.cc:276] 0709 12:24:19.575426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.576427 (+  1001us) service_pool.cc:224] Handling call
0709 12:24:19.630622 (+ 54195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.920699 (+290077us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:20.830505 (+909806us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.037087 (+206582us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.037131 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:21.037179 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.510573 (+473394us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54001f6b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:21.510636 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.510779 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1405,"prepare.queue_time_us":858939,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":49000,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":4881,"spinlock_wait_cycles":669120512,"thread_start_us":1115618,"threads_started":3,"wal-append.queue_time_us":215365}
I20260709 12:24:21.511817 31065 pending_rounds.cc:107] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:24:21.524928 31065 raft_consensus.cc:2889] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:21.525554   622 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.525974   622 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.257879 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1822) took 1921 ms. Trace:
I20260709 12:24:21.222851 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:21.227573   627 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.257519   760 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4.
I20260709 12:24:21.514477 30992 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:21.253575   710 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:21.539003 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.511351 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:21.518250 30989 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.530391 31136 rpcz_store.cc:276] 0709 12:24:19.336856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.336973 (+   117us) service_pool.cc:224] Handling call
0709 12:24:21.257722 (+1920749us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540029ead4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:21.257825 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462592384}
I20260709 12:24:21.513393 31033 leader_election.cc:403] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:21.271008   722 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.544559 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.541734 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.230086   822 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.266827 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.229638   423 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:21.229990   726 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.515947 30986 raft_consensus.cc:2368] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4.
I20260709 12:24:21.255098   830 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.537683 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.538551 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:21.539727 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 309803us
I20260709 12:24:21.540400 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.512596   823 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.252580   835 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:21.251106   735 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:21.259116   480 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:21.512591 32645 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:21.542407 31142 raft_consensus.cc:2410] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:21.542133 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.530885 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:21.754727 30987 raft_consensus.cc:2368] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4.
I20260709 12:24:21.543562 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.515385   853 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.513881   847 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.518627   765 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.531446   433 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:21.534266   627 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.781467   433 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.783479   433 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.580641   722 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.784155   722 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.576443 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:21.518989   709 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.584321 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:21.787393 31694 raft_consensus.cc:2410] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:21.550508 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:21.576421 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 65125.8us
I20260709 12:24:21.537295 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:21.789284 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:21.553578 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 321511us
I20260709 12:24:21.788568 30991 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.575865 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 616) took 2385 ms. Trace:
I20260709 12:24:21.790601 31695 rpcz_store.cc:276] 0709 12:24:19.190505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.191530 (+  1025us) service_pool.cc:224] Handling call
0709 12:24:19.195370 (+  3840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.210264 (+ 14894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.210296 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:19.210692 (+   396us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.395166 (+184474us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:19.395203 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.395261 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.155244 (+1759983us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:21.575571 (+420327us) raft_consensus.cc:1685] finished
0709 12:24:21.575596 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.575804 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255,"raft.queue_time_us":7325,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":3338,"spinlock_wait_cycles":12292480,"thread_start_us":111957,"threads_started":1,"wal-append.queue_time_us":332400}
I20260709 12:24:21.791756 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.534350 30985 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
W20260709 12:24:21.792654 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.554159 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:21.742954 30989 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:21.584507 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:21.793944 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.794914 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.795612 30989 raft_consensus.cc:2410] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:24:21.744745 31664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.014s
W20260709 12:24:21.796168 31664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.003s	sys 0.015s
W20260709 12:24:21.796455 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 606806us
I20260709 12:24:21.788697 30992 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:21.790409 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:21.798458 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.581205 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:21.552990 32654 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.799713 32654 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.538348 31059 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:21.532270   633 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:21.765455 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:21.803220 30989 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:21.764022 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.803709 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.771301 30984 raft_consensus.cc:2410] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:24:21.588145   837 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:21.586339 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:21.605816   801 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 2.036s	user 0.013s	sys 0.022s
I20260709 12:24:21.583963 31689 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:21.580132   739 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.772145   835 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.773409 32645 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.580425   848 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.773882   497 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:21.582549 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:21.774518 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:21.774693 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:21.891891 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 83121.3us
I20260709 12:24:21.892019 30983 raft_consensus.cc:2410] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:21.597594   772 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.775401 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.775995 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.844024   850 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.777097 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:21.581691 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:21.770960   453 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:21.546912   446 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.763413   726 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:21.778748 30889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 311) took 12961 ms (client timeout 15000 ms). Trace:
I20260709 12:24:21.753705   423 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.779088   847 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:21.547017   579 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.574479   710 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.780912   765 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:21.655408 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.802075 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.804253 31063 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:21.916531 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.917124 31063 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:21.917920 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.918831 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.804358 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:21.805711   838 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:21.805781 30992 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:21.806435 31143 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:21.806567   766 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:21.786109   722 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:21.807137 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 253062us
I20260709 12:24:21.808404 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:21.949507 31062 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
W20260709 12:24:21.808945 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 43577.1us
I20260709 12:24:21.545433 32721 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.534336   762 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.764148   841 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:21.967434 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 74920.9us
I20260709 12:24:21.605551   843 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:21.576634   814 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:21.574357   760 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:21.907102   446 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.772351 31137 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:21.771441   735 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.574394   842 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:21.762537   823 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:21.873951   709 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.604776   845 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.884306 32654 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.776537 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:21.894570 31136 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
W20260709 12:24:21.899529 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 122519us
I20260709 12:24:21.900651 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1556) took 2452 ms. Trace:
I20260709 12:24:21.901037   600 raft_consensus.cc:2764] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:21.881944   633 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:21.904181 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:21.898553 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 108215us
I20260709 12:24:21.891040   480 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.897238   850 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.972566   841 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.919999 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:21.887167   835 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:21.998569   856 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
I20260709 12:24:21.895160   848 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.951315 31063 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:21.951949 31140 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:21.954514 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 446) took 2785 ms. Trace:
I20260709 12:24:21.964561 32721 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.894325   497 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:21.905440 30889 rpcz_store.cc:269] 0709 12:24:08.817850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.818013 (+   163us) service_pool.cc:224] Handling call
0709 12:24:21.778683 (+12960670us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:17.826326 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:17.873532 (+ 47206us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:17.873583 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:24:17.873604 (+    21us) tablet.cc:662] Decoding operations
0709 12:24:17.874057 (+   453us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:17.874122 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:17.874149 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:24:17.879448 (+  5299us) tablet.cc:701] Row locks acquired
0709 12:24:17.879490 (+    42us) write_op.cc:260] PREPARE: finished
0709 12:24:17.879646 (+   156us) write_op.cc:270] Start()
0709 12:24:17.879780 (+   134us) write_op.cc:276] Timestamp: P: 1783599857879584 usec, L: 0
0709 12:24:17.879802 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:24:17.886724 (+  6922us) log.cc:844] Serialized 923 byte log entry
0709 12:24:18.846254 (+959530us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2ee878a0ac 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b
0709 12:24:19.146928 (+300674us) op_driver.cc:464] REPLICATION: finished
0709 12:24:19.196764 (+ 49836us) write_op.cc:301] APPLY: starting
0709 12:24:19.196895 (+   131us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:19.197063 (+   168us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:19.197113 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:19.254531 (+ 57418us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:19.254893 (+   362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:24:19.254982 (+    89us) write_op.cc:312] APPLY: finished
0709 12:24:19.255304 (+   322us) log.cc:844] Serialized 29 byte log entry
0709 12:24:19.255536 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:19.255663 (+   127us) write_op.cc:454] Released schema lock
0709 12:24:19.255764 (+   101us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:19.256232 (+   468us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:19.256275 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":453811,"reported_tablets":18,"spinlock_wait_cycles":241536,"child_traces":[["op",{"apply.queue_time_us":49487,"apply.run_cpu_time_us":3012,"apply.run_wall_time_us":59447,"num_ops":2,"prepare.queue_time_us":163294,"prepare.run_cpu_time_us":33993,"prepare.run_wall_time_us":1167050,"raft.queue_time_us":79033,"raft.run_cpu_time_us":10202,"raft.run_wall_time_us":117532,"replication_time_us":1233468,"spinlock_wait_cycles":42395264,"thread_start_us":1221903,"threads_started":4,"wal-append.queue_time_us":1087882,"wal-append.run_cpu_time_us":23271,"wal-append.run_wall_time_us":1295804}]]}
I20260709 12:24:21.968109 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:21.947232   769 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:21.947845   855 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:21.979038 31062 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:21.914975   579 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.985817 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:21.986024 31065 rpcz_store.cc:276] 0709 12:24:19.447952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.450374 (+  2422us) service_pool.cc:224] Handling call
0709 12:24:19.489653 (+ 39279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.525447 (+2035794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.900438 (+374991us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54003962d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:21.900598 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3825131392}
I20260709 12:24:21.972920   814 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:21.980214   433 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.914276   868 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:22.003211 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52377.9us
I20260709 12:24:22.004256   480 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:22.005954 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.882990   739 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:21.981909   735 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.057247 31068 rpcz_store.cc:276] 0709 12:24:19.169498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.169950 (+   452us) service_pool.cc:224] Handling call
0709 12:24:19.170310 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.170862 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.170899 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:19.171397 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.417335 (+245938us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:19.417373 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.417436 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.605083 (+2187647us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:21.948789 (+343706us) raft_consensus.cc:1685] finished
0709 12:24:21.948813 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.954430 (+  5617us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":71,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":10584960,"thread_start_us":245589,"threads_started":1,"wal-append.queue_time_us":398280}
W20260709 12:24:22.056033 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 50449.3us
W20260709 12:24:22.002820   806 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.871s	user 0.013s	sys 0.011s
W20260709 12:24:22.005231   839 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 12:24:22.006418 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 86504.3us
W20260709 12:24:22.061582 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 93533.3us
I20260709 12:24:21.985817   851 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.004912   850 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:22.065683 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 79952.2us
I20260709 12:24:22.006507 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:22.068133 32645 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.068333   868 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:22.068714   845 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:22.069434   739 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.069522   627 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.071236 31138 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 12:24:22.071292   579 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.069439   480 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.072057 31059 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:22.073145 30991 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:22.025799   622 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.003618   760 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:24:22.024576   841 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.006282   838 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:24:22.004717   633 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.006615   822 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.258917 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.261477 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.079432 30990 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:22.444222 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1784) took 1213 ms. Trace:
I20260709 12:24:22.261399 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.212036 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.239120 32654 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.409058 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.260648 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.264030 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:22.264221 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.243119 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1861) took 1088 ms. Trace:
I20260709 12:24:22.076630 32721 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.236505 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1869) took 1044 ms. Trace:
I20260709 12:24:22.061707   814 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.450032 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 1866) took 1282 ms. Trace:
I20260709 12:24:22.466643 31140 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:22.458917 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.452361 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.451270 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:22.513795 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:22.512288 31061 rpcz_store.cc:276] 0709 12:24:21.167454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.211695 (+1044241us) service_pool.cc:224] Handling call
0709 12:24:22.449975 (+238280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:22.514518 31068 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:22.464510 31144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1475) took 3084 ms (client timeout 1520 ms). Trace:
I20260709 12:24:22.459270 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.463665 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.445897 31697 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:22.515965 31067 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:22.515502 31144 rpcz_store.cc:269] 0709 12:24:19.379908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.439221 (+ 59313us) service_pool.cc:224] Handling call
0709 12:24:22.464343 (+3025122us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b540001c054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:22.464442 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590218112}
I20260709 12:24:22.456176 31694 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:22.277947 30888 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:22.275677 31106 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.66:45157 timed out after 1.764s (SENT) [suppressed 8 similar messages]
I20260709 12:24:22.465233 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.517602 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.511575 31064 rpcz_store.cc:276] 0709 12:24:21.192397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.509149 (+316752us) service_pool.cc:224] Handling call
0709 12:24:21.509400 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.533379 (+ 23979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.533420 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.533460 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:21.533486 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.236242 (+702756us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b5400383954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:22.236288 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.236430 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54695808}
I20260709 12:24:22.450459 32654 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.448770 30991 rpcz_store.cc:276] 0709 12:24:21.230934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.576218 (+345284us) service_pool.cc:224] Handling call
0709 12:24:22.444146 (+867928us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:22.521512 31140 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:22.522962 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 516499us
I20260709 12:24:22.523554 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:22.524070 31694 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.519544 31136 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:22.526621 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.518321 30888 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:22.526629 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.528833 30888 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 5, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 5 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:22.528851 31144 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:22.530418 30888 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 3, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 3 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:22.531065 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.531270 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.531857 31067 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:22.532333 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.533739 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:22.534859 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:22.536039 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.458647 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.522404 31060 rpcz_store.cc:276] 0709 12:24:21.154893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.155008 (+   115us) service_pool.cc:224] Handling call
0709 12:24:21.160843 (+  5835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.189395 (+ 28552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.189441 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.189956 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.238486 (+ 48530us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:21.238536 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.238622 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.242873 (+1004251us) raft_consensus.cc:1685] finished
0709 12:24:22.242900 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.243050 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28141,"raft.queue_time_us":30118,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":48218,"threads_started":1,"wal-append.queue_time_us":352444}
W20260709 12:24:22.154040   804 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 2.329s	user 0.004s	sys 0.012s
I20260709 12:24:22.531879 30888 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:22.074106 31690 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:24:22.448307 31033 connection.cc:256] RPC call timeout handler was delayed by -2.3397691700001815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:24:22.546283 32654 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:22.543594 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:22.545431 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.546358 31033 connection.cc:256] RPC call timeout handler was delayed by -2.0136238969998885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2515s for a response
W20260709 12:24:22.547351 31033 connection.cc:256] RPC call timeout handler was delayed by -1.9197402879999572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260709 12:24:22.547650 31033 connection.cc:256] RPC call timeout handler was delayed by -1.8213691700002528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
W20260709 12:24:22.547919 31033 connection.cc:256] RPC call timeout handler was delayed by -1.7698137849997693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 12:24:22.548192 31033 connection.cc:256] RPC call timeout handler was delayed by -1.7551908020000155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 12:24:22.548450 31033 connection.cc:256] RPC call timeout handler was delayed by -1.7223691700000927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
W20260709 12:24:22.548727 31033 connection.cc:256] RPC call timeout handler was delayed by -1.6390355409998847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 12:24:22.549002 31033 connection.cc:256] RPC call timeout handler was delayed by -1.6307035719996748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
I20260709 12:24:22.078332 31137 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:22.263312 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.553771   845 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:22.554540   627 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.554720 32721 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:22.555186 32721 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.554813   868 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.555572 32721 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.252319   453 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.196575   874 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.391047   604 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:22.550516 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:22.550642 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 90520.3us
I20260709 12:24:22.551292 31058 raft_consensus.cc:2368] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
I20260709 12:24:22.438898   735 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:22.553944 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.779980 31060 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
W20260709 12:24:22.550060 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 286071us
I20260709 12:24:22.780512   819 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.781008   819 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.781550 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:22.541085 30888 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:22.785045   729 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:22.785424   729 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.784878 30888 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 5, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 5 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:22.785744   729 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.786270 30888 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:22.783974   819 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.274528   446 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:22.078699   772 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.296521   848 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.236156   726 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.079653   823 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.790196   825 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.258908 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.211659   762 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.260262   433 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:22.235296   599 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.196575   710 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.196337   872 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:22.445374   878 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.254860   497 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.215266   423 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:22.363080 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.445391   818 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.769932 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:22.601998 31033 connection.cc:256] RPC call timeout handler was delayed by -1.5340137849998428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 12:24:22.561896   453 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:22.764858   886 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.572026   604 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:22.801683   604 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.766862 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:22.465870   827 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.803150   827 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.803632 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.804822 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.549731   871 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:22.196908   853 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.805402 31690 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:22.805811   871 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.806481   886 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.807080 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.807423 30988 raft_consensus.cc:2368] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:22.599078 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.807869 31691 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:22.807942   871 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.808081 31692 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:22.808661   886 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:22.809180 31033 connection.cc:256] RPC call timeout handler was delayed by -1.5117527329998666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
I20260709 12:24:22.809089 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:22.809654 31033 connection.cc:256] RPC call timeout handler was delayed by -1.3134902879996844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
I20260709 12:24:22.557662   842 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:22.810424   842 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:22.810942   842 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.811079 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 629) took 1228 ms. Trace:
I20260709 12:24:22.811539 31693 rpcz_store.cc:276] 0709 12:24:21.582673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.582797 (+   124us) service_pool.cc:224] Handling call
0709 12:24:21.583088 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.985976 (+402888us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:22.026526 (+ 40550us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.557466 (+530940us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.562467 (+  5001us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:22.571544 (+  9077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.641534 (+ 69990us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b18003dcc60. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:24:22.810829 (+169295us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b5400512a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:22.810892 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.811020 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87240,"prepare.queue_time_us":27400,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":99135,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":916488448,"thread_start_us":308310,"threads_started":1,"wal-append.queue_time_us":517948}
I20260709 12:24:22.597581   627 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.813979 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.814595   627 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.814965 31693 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 1.
I20260709 12:24:22.815920 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.816593   627 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.817029 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.817721   853 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.809989 31692 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:22.818140 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.818282   818 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.819041 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.819868 31692 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:22.557636   579 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:22.797698   872 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:22.629931 32721 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.826567 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:22.794019   814 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.827090 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.827920 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.828220 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.828416 32721 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:22.828692 31693 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:22.551396   480 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:24:22.828967 30987 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:22.829356 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.767683 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:22.542673 31059 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 1865) took 1354 ms (client timeout 1503 ms). Trace:
I20260709 12:24:22.599870   874 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:22.623957   847 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.705s	user 0.007s	sys 0.002s
I20260709 12:24:22.596918 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.610488   730 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:22.842015 31664 connection.cc:256] RPC call timeout handler was delayed by -1.2041829389995655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 12:24:22.826620   579 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:22.843057   579 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.841650   453 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.828694   814 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.846298   729 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.846634 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:22.847146   453 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.848074 31141 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:22.848398   729 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:22.849553 31065 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:22.644551   822 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:22.610932   835 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:22.762578 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 226658us
W20260709 12:24:22.778386 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 327176us
I20260709 12:24:22.766048 31694 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:22.866570 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:22.778950 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 503272us
I20260709 12:24:22.790252 31661 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:22.787761 30888 catalog_manager.cc:5697] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 4, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:22.799538 31697 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:22.907177 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:22.666281 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.910261   762 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:22.813984 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.911263 31139 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:22.912541 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.913244 31139 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:22.913444 32654 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.915123   822 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:22.922767 32654 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.915843   599 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:24:22.924435 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.918637 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:22.925222 31138 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 3: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:22.829511 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.921962 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.830869   423 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.798040 31689 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:22.831043 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:22.797205   433 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.820470   818 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:22.810010 31033 connection.cc:256] RPC call timeout handler was delayed by -1.2673030910000307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 12:24:22.826750   872 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:22.941670 31064 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:22.709774   850 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:22.835969 31692 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
W20260709 12:24:22.836732 31059 rpcz_store.cc:269] 0709 12:24:21.164135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.190185 (+ 26050us) service_pool.cc:224] Handling call
0709 12:24:22.518870 (+1328685us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:22.941444 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 157746us
I20260709 12:24:22.837200   622 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:22.838759   874 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:22.838867   842 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.839123   847 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:22.850189 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:22.857775 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.860002 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.963485 31068 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:22.963704 31060 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:22.860615 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.861032 31698 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:22.862468   878 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:22.906198 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.965531 31067 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:22.966385 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:22.910364   853 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:22.915482   497 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:22.925787 31693 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:22.830084 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.682659 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1778) took 1498 ms. Trace:
I20260709 12:24:22.834354 31062 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:22.920634 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:22.937556 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.939744 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:22.871078   814 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 12:24:22.909233 30888 catalog_manager.cc:5697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 7, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 7 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:22.940773 31139 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:22.940943 31135 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:22.867733   886 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:22.954530 31661 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:23.211303 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:23.218261 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 263797us
I20260709 12:24:23.218406 31694 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.211779 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 632) took 1628 ms. Trace:
I20260709 12:24:23.219485 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.220180 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:23.091872 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:23.220916 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.221130   730 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.221532   730 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.221856   730 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:23.222041 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.222976 30986 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:23.218843 30991 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:23.225248 31694 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:23.226274 30984 rpcz_store.cc:276] 0709 12:24:21.184332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.184431 (+    99us) service_pool.cc:224] Handling call
0709 12:24:22.680262 (+1495831us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54003462d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:22.682554 (+  2292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347296128}
I20260709 12:24:22.959337   729 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:23.149024 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:23.227345 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 135574us
I20260709 12:24:23.235242 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:23.236053 31141 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:23.236965 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:23.237812 31067 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:23.078013 30990 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1782) took 1892 ms (client timeout 2125 ms). Trace:
W20260709 12:24:23.239027 30990 rpcz_store.cc:269] 0709 12:24:21.185443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.770564 (+585121us) service_pool.cc:224] Handling call
0709 12:24:23.077783 (+1307219us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54000235d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:23.077944 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644462976}
W20260709 12:24:23.211328 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 361188us
W20260709 12:24:23.052408 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1794) took 1795 ms (client timeout 2065 ms). Trace:
I20260709 12:24:23.240295 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:23.240427 30987 rpcz_store.cc:269] 0709 12:24:21.256584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.771696 (+515112us) service_pool.cc:224] Handling call
0709 12:24:23.052224 (+1280528us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b540029b8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:23.052354 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940710784}
W20260709 12:24:23.053202 30988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1789) took 1798 ms (client timeout 1601 ms). Trace:
I20260709 12:24:23.240764 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:23.241325 30988 rpcz_store.cc:269] 0709 12:24:21.255142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.507753 (+252611us) service_pool.cc:224] Handling call
0709 12:24:23.053004 (+1545251us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400024ed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:23.053138 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2986804608}
W20260709 12:24:23.230682 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 310129us
I20260709 12:24:23.231516 30991 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:23.242504 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:22.961925   752 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:22.943933   423 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.243340   885 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:24:23.243283   848 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:23.244277   848 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.244408 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.244696   848 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:23.245038 30992 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:23.245467   729 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:23.194402 31058 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 1860) took 2049 ms (client timeout 1764 ms). Trace:
I20260709 12:24:22.953519   622 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:23.219246 31696 rpcz_store.cc:276] 0709 12:24:21.583519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.587492 (+  3973us) service_pool.cc:224] Handling call
0709 12:24:21.587746 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.779897 (+192151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.779945 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.780003 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:21.780053 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.211528 (+1431475us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54006a1654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:23.211576 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.211709 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441318656}
I20260709 12:24:23.242380 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:23.227315 31697 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
W20260709 12:24:23.247611 31058 rpcz_store.cc:269] 0709 12:24:21.145201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.145328 (+   127us) service_pool.cc:224] Handling call
0709 12:24:23.194122 (+2048794us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002a4c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:23.194315 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3129636224}
I20260709 12:24:23.235524 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:23.243270   822 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.233728   599 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.228358 30888 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:24:23.326937 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 75101.9us
I20260709 12:24:23.048681 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:23.372393 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:23.373236 31142 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:23.375108 30888 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:23.377892 30888 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:23.311255   627 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:23.247766   842 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.211567   853 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.408717 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:23.410351 31064 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.410276 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:23.313653 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1835) took 1053 ms. Trace:
I20260709 12:24:23.411499 31140 rpcz_store.cc:276] 0709 12:24:22.259823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.513195 (+253372us) service_pool.cc:224] Handling call
0709 12:24:22.513468 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.522021 (+  8553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.313333 (+791312us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b54001e9bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:23.313578 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17814784}
I20260709 12:24:23.231860   604 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.019315   769 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:23.034657 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:22.960005   480 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.287s
W20260709 12:24:23.087419 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 147016us
I20260709 12:24:23.381170 30983 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:23.243430   752 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.234074   886 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.389107 31067 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:23.490864 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:23.491036 32654 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:23.480926   823 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.492079   823 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.494589 31138 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:23.491420 31067 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.074604   850 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:23.498822 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 256540us
I20260709 12:24:23.404053 30985 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:23.501395 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:23.502142 30985 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:23.416889 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:23.295574 31139 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:23.211584 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:23.250998   872 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.310837   497 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.486083   752 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.486902   864 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:23.250801   835 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:23.515508 31067 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:23.386765   822 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.099s
W20260709 12:24:23.518571 31033 leader_election.cc:336] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.601s (SENT)
I20260709 12:24:23.482115 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.526657 30985 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:23.527662 30992 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:23.527791   729 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.385236   627 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.560958 31695 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:23.452384   847 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.490080   886 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:23.540270 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:23.495290   823 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.420028   853 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.558456 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:23.555018 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:23.561235 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.568605 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.571738 30992 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:23.558079   884 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.450043   622 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.572417   884 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.572645 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.573554 32721 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:23.573951 32721 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:23.574332 32721 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.574463   884 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.541124   864 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:23.541396   835 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:23.575940 30983 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:23.571858 31065 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:23.579921 30983 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:23.572664   622 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.585986   886 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:23.591172 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 72644.4us
I20260709 12:24:23.595196   497 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:23.595624   853 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.600855 32721 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.602201 30992 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:24:23.603156 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.606837 31065 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:23.583065 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.608886 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.614934 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:23.616299 31062 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:23.616549 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:23.565455   729 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:24:23.627703 31065 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:23.629626   730 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.577306   835 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.577865   864 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:23.639086 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 310750us
I20260709 12:24:23.643831 31064 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:23.646175 31062 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:23.619131   890 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 12:24:23.669513 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 168176us
W20260709 12:24:23.635572   433 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.677s	user 0.003s	sys 0.006s
I20260709 12:24:23.671602 30989 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:23.671643 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 55185.8us
W20260709 12:24:23.673107 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.685906 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.671092 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:23.675273 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.686513 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.686980 30985 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:23.687760 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.688257 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.686518 30988 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:23.627980   872 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:23.677466 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.678062 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.682188   600 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.683583 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.681258   841 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.693037 30988 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:23.693961 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.711809 30989 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:23.637053   896 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:23.697926 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.705888 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.723990 30987 raft_consensus.cc:2376] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5. Re-sending same reply.
I20260709 12:24:23.691840 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:23.707752 30983 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:23.724921 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.725291 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.708701 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:23.709444 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.710036   600 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.712180 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.732894 30988 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:23.733786 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.734527 30988 raft_consensus.cc:2393] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:23.735188 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.736075 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:23.736932 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.682669   847 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.682971   627 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:23.731407 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:23.692657   729 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:23.737926   847 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.738744 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.747586 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.748353 30989 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:23.741564 30984 raft_consensus.cc:2410] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:23.742508 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:23.746662 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:23.747673 30992 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:23.749323 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.740572   600 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.682416   848 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.752281 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.753201 30987 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:23.756960 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:23.746695   599 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.763157   884 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:23.763998   884 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:23.764161   847 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.765512   730 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.773594   600 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.773726 31067 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:24:23.774343 31067 pending_rounds.cc:85] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 2
I20260709 12:24:23.774650 31067 pending_rounds.cc:107] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:24:23.774966 31067 raft_consensus.cc:2889] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:23.733176   872 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:23.777410 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.779970 30989 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:23.754241   853 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.725759   841 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.798568 30989 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:23.704808   433 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.801759 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.779387   872 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:23.801781 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:23.816567 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 201741us
I20260709 12:24:23.817292 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:23.818395 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:23.822912 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 117090us
I20260709 12:24:23.824110 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:23.830017 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 87581.2us
I20260709 12:24:23.817054   872 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:23.829582   627 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.836010 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.836505 30986 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:23.831180 30985 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:23.830633   622 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:23.839826   622 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.840296   622 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:23.845450   872 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.836967   627 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.848176 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:23.850068 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:23.850091 31694 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:23.854020 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:23.865655 31694 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:23.867312 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:23.854148   627 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:23.869127 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:23.869815 30986 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:23.875046 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 118151us
I20260709 12:24:23.878070 30986 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:23.882402 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:23.879307 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:23.880237   729 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:23.888664 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:23.892560 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:23.897631 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 43689us
I20260709 12:24:23.898337 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:23.900055 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:23.901087 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 715) took 1123 ms. Trace:
I20260709 12:24:23.901503 31690 rpcz_store.cc:276] 0709 12:24:22.777726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.806190 (+ 28464us) service_pool.cc:224] Handling call
0709 12:24:23.899868 (+1093678us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:23.901009 (+  1141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13353216}
I20260709 12:24:23.867409 32721 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.881079 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:23.890607 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:23.889357   909 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:24:23.911194   905 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:24:23.935120   433 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:23.935497 31689 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:23.936663 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:23.938921   903 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:24:23.909909   913 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.940076   913 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:23.940451   913 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:23.942428   912 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.943130   912 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:23.943583   912 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.944296   886 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:23.944979   886 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:23.945703   912 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.946388   729 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.947665   909 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:23.948720   909 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.956362   423 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:23.958673   845 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:23.959185   845 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:23.928200   910 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.959892   910 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:23.960261   845 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:23.966533 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:23.967070 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:23.967396 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:23.968722 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:23.969348 31698 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:23.968346 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 70114.7us
I20260709 12:24:23.970646 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:23.919421   908 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.978720 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:23.986764 31696 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:23.984283 31698 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:23.927968 31693 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:23.908293   904 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
W20260709 12:24:23.995602 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 104594us
I20260709 12:24:23.961705   845 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:23.969314   912 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:23.950042 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 724) took 1153 ms. Trace:
I20260709 12:24:24.001057 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:23.995083   848 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.013384   872 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.014902   917 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:24.015560   917 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:24.015955   917 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:24.017252 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 109949us
I20260709 12:24:24.018364 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.019975 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:23.930402   752 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.021569 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:24.032042 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:24.032718 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.006018   910 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:23.946122   886 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.039114   898 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:24:24.022027   853 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:24.045990   907 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:24:24.035005 31689 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:24.033129   854 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:24.140719 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 139738us
I20260709 12:24:24.050755   898 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:23.997521   864 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.043591   912 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.037465   730 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:24.090770   896 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:24.043942   627 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.014922   835 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.135396 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1887) took 1267 ms. Trace:
I20260709 12:24:24.159787 31067 rpcz_store.cc:276] 0709 12:24:22.868348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.560480 (+692132us) service_pool.cc:224] Handling call
0709 12:24:23.560845 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.775519 (+214674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.135145 (+359626us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400383454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:24.135332 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488381312}
I20260709 12:24:24.140715 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 710) took 1365 ms. Trace:
I20260709 12:24:24.161307 31695 rpcz_store.cc:276] 0709 12:24:22.775427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.775595 (+   168us) service_pool.cc:224] Handling call
0709 12:24:22.775974 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.561367 (+785393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.140463 (+579096us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54005811d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:24.140641 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1804897152}
I20260709 12:24:24.127213   904 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:24.145550   925 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:24.170181 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58470.9us
I20260709 12:24:24.146096   898 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.179461 31694 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:24.159252   730 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.170185   925 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.174957 31690 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:24.188349   921 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:24.186695   929 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:24:24.188118   926 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:24:24.193151   922 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:24:24.197386   622 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.162974   599 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:24:24.003415 31691 rpcz_store.cc:276] 0709 12:24:22.796809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.831815 (+ 35006us) service_pool.cc:224] Handling call
0709 12:24:22.940611 (+108796us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:22.952021 (+ 11410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:22.952056 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:22.952580 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:24:23.325314 (+372734us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:23.377248 (+ 51934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.389768 (+ 12520us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:23.503860 (+114092us) raft_consensus.cc:1685] finished
0709 12:24:23.503890 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.949666 (+445776us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540034fdd0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:23.949960 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249405312,"thread_start_us":282560,"threads_started":1,"wal-append.queue_time_us":421669}
I20260709 12:24:24.205717 31141 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:24:24.207255 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:24.208169 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:24.209873   730 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.215183 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.190616   898 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.235144 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.244879 31696 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:24.221880 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.192183   925 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.241014 30989 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:24.248380 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:24.224512 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.234910 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.242493   752 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.236326 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.252476 30987 raft_consensus.cc:2376] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4. Re-sending same reply.
I20260709 12:24:24.202378 31693 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:24.214790 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:24.204901   889 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.166s	user 0.009s	sys 0.005s
I20260709 12:24:24.245982 30986 raft_consensus.cc:2368] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4.
I20260709 12:24:24.256841 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.257282 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.238814 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.258949 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.251547 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.255049 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:24.260322 30989 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:24.261351 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.221660 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.265196 30992 raft_consensus.cc:2368] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:24.266323   929 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.263286   884 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:24.266911   929 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.267822   884 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:24.268278   884 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.268473   934 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.268994   934 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.239089   851 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:24.273377 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.247525   627 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:24.275215   929 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:24.279115 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.280647 30987 raft_consensus.cc:2376] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3. Re-sending same reply.
W20260709 12:24:24.283375 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.264462   845 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:24.263756 31143 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:24.269418   884 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.243883   913 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.272761   851 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:24.294904   851 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.295464 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:24.296067   864 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.298310   497 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:24.299382   497 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:24.299731   497 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.300581   497 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.301802   497 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.204756   896 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.290809   929 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.289732   934 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.310321   884 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:24.311614   912 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.312686   814 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:24.323791 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.307533   904 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:24.344040 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.351234   934 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:24.353040 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.357838 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.359859 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:24.327651   929 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.351787   934 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.365895 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:24.368789   939 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:24.369268   939 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:24.369619   939 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:24.370323   942 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:24.371613 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.375288 31064 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:24.376134 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.377336 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.358587   627 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:24.379392 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.379961 31065 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:24.380702 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.380749   841 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.381680 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.382189 31065 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:24.375595   622 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:24.388082 31068 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:24.390900 31064 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:24.393010 31064 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:24.395941   925 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.397100 31067 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:24.391829   934 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.402983 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.405794 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.403551 31060 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:24.370784   913 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:24.414357 31065 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:24.418210 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.391578 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.420356 31060 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
W20260709 12:24:24.433739 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 73984.1us
I20260709 12:24:24.405176   925 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:24.441404 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.442538 31065 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:24.443801 30987 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
W20260709 12:24:24.448277 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.450832 31065 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:24.451524 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.448704 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:24.495873 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54550.7us
W20260709 12:24:24.496464 31106 leader_election.cc:336] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Timed out: RequestConsensusVote RPC to 127.29.37.66:45157 timed out after 1.503s (SENT)
I20260709 12:24:24.495033 31067 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.497248 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.449970 31059 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.501370 31060 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:24.382316   904 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:24.503156 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54573.2us
I20260709 12:24:24.504417 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:24.504155   925 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
W20260709 12:24:24.507550   941 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:24.504307   934 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:24.498059 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.509285   497 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:24.509820 31065 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 2: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:24.510070   841 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:24.514513 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.498806 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.515408 31060 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:24.518431   497 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.519608   497 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.520258   934 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:24.522521   841 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.523720   841 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.525653 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:24.526180   827 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.526703   827 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:24.526779 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.527124   827 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:24.527552   827 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.528182 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:24.528884   934 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:24.529253   934 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:24.529613   934 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.426477   917 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.531669 31061 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:24.532513   917 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:24.535326 31059 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:24.535748   752 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:24.536042 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.537326 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.567230 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.538611   934 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.547020 31060 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:24.559973 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1885) took 1700 ms. Trace:
I20260709 12:24:24.568308 31058 rpcz_store.cc:276] 0709 12:24:22.859051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.558950 (+1699899us) service_pool.cc:224] Handling call
0709 12:24:24.559333 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.559547 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.559584 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.559643 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:24.559684 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.559756 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.559903 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:24.569887 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.570649 31058 raft_consensus.cc:2376] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7. Re-sending same reply.
I20260709 12:24:24.496863   886 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:24.564850 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58106.7us
I20260709 12:24:24.495936   842 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:24.572185 31067 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:24.575956 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.579268 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.581378 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:24.583369 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.586347 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:24.585346 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.577298   842 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.638489 31697 raft_consensus.cc:2368] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
I20260709 12:24:24.641220 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:24.641322 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56072.2us
I20260709 12:24:24.641911 31661 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.577939   627 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:24.682169 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:24.699584 30992 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:24.646338   847 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:24.701459 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.694818   769 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:24.702832 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.705901   845 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:24.706408   845 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.707409   845 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.586199   917 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:24.711689 31135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1485) took 3024 ms (client timeout 1660 ms). Trace:
I20260709 12:24:24.711941 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.639657   925 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:24.684327 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.716606   934 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:24.578030   944 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.717082   851 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:24.717461 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.696125 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.717130   842 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.718294   842 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:24.718462 31109 connection.cc:256] RPC call timeout handler was delayed by -1.0115186179996272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260709 12:24:24.718859 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1928) took 1016 ms. Trace:
I20260709 12:24:24.719285   818 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.719223 30984 rpcz_store.cc:276] 0709 12:24:23.702389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.709176 (+  6787us) service_pool.cc:224] Handling call
0709 12:24:24.718693 (+1009517us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b54003462d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:24.718810 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20490496}
I20260709 12:24:24.719897   818 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.649766 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.721643   884 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:24.638289   633 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:24.722056   884 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.722656   433 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.666824 31106 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:24.638440   622 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:24.712100 31135 rpcz_store.cc:269] 0709 12:24:21.687587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.747410 (+ 59823us) service_pool.cc:224] Handling call
0709 12:24:24.711522 (+2964112us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000942d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:24.711610 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607751040}
I20260709 12:24:24.728194 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:24.728693 31698 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 2.
I20260709 12:24:24.712821 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:24.713835 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:24.729555 31695 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:24.714072 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.730206 31141 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:24.721961 31065 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:24.651098   913 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:24:24.723776   884 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.581547   752 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:24.722580 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.738608 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.687778   600 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:24:24.643759   886 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:24.736109 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.650849 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.736408   847 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.739607 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:24.744057 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.735806 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.744736 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:24.709007   845 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:24.744583 31698 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:24.748361 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:24.750473 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.639130 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.735553 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:24.709682 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.746573 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:24.722111   818 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:24.759143 31695 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.759166 31141 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:24.760763 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.736985   633 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.762867 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:24.765933   633 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:24.766010 31144 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:24.739749   898 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.763523 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.764851 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:24.774837 30991 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:24:24.775194 30991 pending_rounds.cc:85] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 3
I20260709 12:24:24.775403 30991 pending_rounds.cc:107] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:24:24.775627 30991 raft_consensus.cc:2889] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:24.767742 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.737052   929 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.775498 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.782948 31141 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:24.776775 31135 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:24.766729 31698 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:24.753448   814 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:24.788923 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:24.789944   947 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:24:24.790299 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.644656   930 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.120s
I20260709 12:24:24.758260   885 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:24.758757   622 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:24.740494   433 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.797362 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:24.803910 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.759048 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.808141   929 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.813367 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:24.813905 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:24.815268 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.820222 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:24.783474 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:24.753284   847 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.787629   818 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:24.841603 31144 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:24.842612 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:24.850550 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.763631   939 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.853798 31698 raft_consensus.cc:2376] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7. Re-sending same reply.
I20260709 12:24:24.836396 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.852159 31696 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:24.855527 31142 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:24.852124 30984 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:24.857383 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.859368 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.862964 31144 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:24.858269 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.811728   885 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.811370   841 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.768965   497 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:24:24.836591   929 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.810992   930 consensus_queue.cc:1243] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:24.810628   622 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:24.818058 31137 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:24.838855   900 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.816680   814 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:24.879245 31144 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:24.887403 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:24.867933 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 139317us
I20260709 12:24:24.868737 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.856097 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.873027   841 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.850324   769 raft_consensus.cc:2764] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:24.879802 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.897056 31065 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:24.866911   885 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:24.897733 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.898823 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.869616   929 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:24.899860 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.859745   433 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:24.867417 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.901867 31144 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:24.879166   622 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.900727 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.878629 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:24.907614 31142 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
W20260709 12:24:24.880494 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 213779us
I20260709 12:24:24.910529 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.913190 31135 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:24.913533 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.914254 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.915621 31059 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:24.922994   841 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.925987 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:24.927434 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:24.928755   845 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:24.929405   845 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:24.934619 31690 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:24.934046 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.938727   827 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.941277 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.943305   939 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:24.911872   818 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.944824 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.944980   818 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.947587 30989 raft_consensus.cc:2410] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:24.950642 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.954340 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:24.956771 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.957341 31135 raft_consensus.cc:2376] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5. Re-sending same reply.
I20260709 12:24:24.957197 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.959640 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.960770 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:24.960325 31135 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:24.966115 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:24.969278 31695 raft_consensus.cc:2410] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:24.973307 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:24.944046   814 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:24.962416 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 199518us
I20260709 12:24:24.963060   841 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:24.978143 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:24.978561 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:24.984220 31065 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:24.967761   845 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:24.985160   955 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:24.993556 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1893) took 1954 ms. Trace:
I20260709 12:24:24.993948 31062 rpcz_store.cc:276] 0709 12:24:23.039196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.618045 (+578849us) service_pool.cc:224] Handling call
0709 12:24:23.646017 (+ 27972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.646627 (+   610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.993380 (+1346753us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b5400397bd0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:24.993503 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59633024}
I20260709 12:24:24.996603 31065 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:24:24.975678 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 230110us
I20260709 12:24:24.979060 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:24.993129   955 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.998338   955 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:25.025312 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 47821.9us
I20260709 12:24:24.968111   633 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:24.995800 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:24.986541   845 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.034734 31058 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:24.970315   953 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:24:25.038404 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:25.040906 31068 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:25.047020 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:25.040795 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 128282us
I20260709 12:24:24.967921   433 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 12:24:25.042804 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44888.2us
I20260709 12:24:24.970574   952 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:24:25.044413 31059 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:25.045271 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:25.046169 30985 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:25.049418 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.049167 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:25.055408   943 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.612s	user 0.010s	sys 0.000s
I20260709 12:24:25.035068   822 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:25.040841   925 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:25.059054 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:25.046262   845 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.046087 31143 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.046723   898 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:25.059904   951 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.067s	user 0.001s	sys 0.009s
W20260709 12:24:25.065845 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 168279us
I20260709 12:24:25.059906   600 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:25.054355   917 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:25.089073 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:25.089428 31137 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:25.091372 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.076779 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.093111 31058 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:25.059446   822 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:25.046294   930 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:24:25.053516   954 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:25.095479 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:25.100344 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.064726   939 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:25.101166 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49622.4us
I20260709 12:24:25.065587   847 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:25.105363 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:25.094905   952 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:25.096174   963 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:25.103055   855 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:25.109011   433 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:25.091706   884 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.112417 31144 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:25.102874   917 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:25.116358 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:25.093822   898 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.119817 31691 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:25.116968   841 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:25.123133   841 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:25.106036   939 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:25.126901   939 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:25.129498   956 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:25.130702   960 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:25.133690 31138 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:25.111688   952 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:24:25.126111 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.123930   841 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.105641   969 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:25.114598   963 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:25.152140 31689 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:25.152652   885 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.152522   952 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.144578   969 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.144048   622 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.137148   822 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.105239   954 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.144963   433 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:24:25.155745   841 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.154500   970 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:25.159968   900 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:25.165264 31142 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:25.168463 30988 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:25.166785 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.174001 31141 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:25.165846 31143 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:25.153693   600 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:25.159754   956 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:24:25.161016   917 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.133692   955 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.160858   822 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.166976   952 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.166730   633 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.169821   970 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:25.159171   969 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.168187   885 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.178901   944 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:25.202950 31142 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:24:25.207129 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 107478us
I20260709 12:24:25.168788   929 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.212716 31065 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:25.215799 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1883) took 2357 ms. Trace:
I20260709 12:24:25.216069 31061 rpcz_store.cc:276] 0709 12:24:22.858525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.413030 (+1554505us) service_pool.cc:224] Handling call
0709 12:24:24.413449 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:24.532126 (+118677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.215623 (+683497us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001bb554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:25.215754 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271166848}
I20260709 12:24:25.217012 31061 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:25.219122 31061 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:25.220436 31064 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:25.220525 31142 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:25.220693 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:25.224045 31061 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:25.218973   953 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:24:25.225806   953 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:25.226717   953 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:25.227095   953 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:25.228049   953 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.229482   953 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.229868   884 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.218204   855 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:24:25.219832   898 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.231684   970 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.233194 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.164386   963 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.233894 31139 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.234543   963 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.247200 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:25.296126   600 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:25.321436   938 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.779s	user 0.001s	sys 0.001s
I20260709 12:24:25.327610 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.328563   927 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:25.313237   926 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:25.337344 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.337718 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:25.354641 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1852) took 1106 ms. Trace:
I20260709 12:24:25.355106 31136 rpcz_store.cc:276] 0709 12:24:24.248113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.248245 (+   132us) service_pool.cc:224] Handling call
0709 12:24:24.248510 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:24.248839 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.248867 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:24.249285 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 12:24:24.326819 (+ 77534us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:24.326867 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.326954 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.794470 (+467516us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b4800251958. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef07bdaa1 00007f2ef07b4b37 00007f2ef07b49ca 00007f2ef08ac26f 00007f2ef08abf37 00007f2ef08c055c 00007f2ef08c0508 00007f2ef08c0488 00007f2ef08c0443 00007f2ef08bf6ec 00007f2eef7e9a96
0709 12:24:25.320867 (+526397us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:25.354450 (+ 33583us) raft_consensus.cc:1685] finished
0709 12:24:25.354470 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.354585 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":978,"raft.run_cpu_time_us":2393,"raft.run_wall_time_us":543014,"spinlock_wait_cycles":55374336,"thread_start_us":77176,"threads_started":1,"wal-append.queue_time_us":163355}
I20260709 12:24:25.233206   970 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:25.360285 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 139613us
I20260709 12:24:25.362686 31136 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.364395 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:25.366780 31697 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:25.367472 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:25.371341 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 273253us
I20260709 12:24:25.376271 31139 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:25.379227 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:25.381130 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48047b2b139146c38a821328cf7bd148"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:24:25.371572 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:25.382906 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.381659 31695 raft_consensus.cc:606] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:25.383934 31697 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:25.356065   823 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:25.389292 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.392374 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.389209 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 282831us
W20260709 12:24:25.398478 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.398363   900 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:25.398669   885 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:25.399032   900 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:25.399209   885 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.399957   900 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.401731   885 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.403476   981 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:25.380874   709 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:25.386636   896 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:25.376721   835 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.380769   769 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:25.421068 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50787.3us
I20260709 12:24:25.390960   872 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:25.397074   855 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:25.463379 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 80553.6us
I20260709 12:24:25.421916   709 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:25.449551   896 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:25.449957   769 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:25.470157   709 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:25.458753   855 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:25.481549   855 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.397269   823 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.492622 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:25.478283   896 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:25.488384   818 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.523s	user 0.014s	sys 0.007s
I20260709 12:24:25.482559   769 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:25.500573 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 49543.1us
I20260709 12:24:25.489254   872 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:25.492225   929 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:25.500710   729 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:25.579562   823 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.512346   818 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.589020 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:25.514500   988 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 12:24:25.589875   729 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:25.537482   845 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.601447 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.603137 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:25.604254   983 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:24:25.604343 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.588626   939 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.612092 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.612990 31140 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:25.615692 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:25.623994 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.611976   991 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:24:25.635715 31136 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:25.639650 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:25.640740 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:25.601009   872 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.642575   729 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:25.627061   978 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:24:25.651176 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:25.670413 31140 raft_consensus.cc:2410] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:24:25.651540   985 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:25.618892   823 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:25.655740   845 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.656764   769 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:25.657809   990 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:24:25.658058 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1588) took 1289 ms. Trace:
I20260709 12:24:25.686661 31063 rpcz_store.cc:276] 0709 12:24:24.368433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.371986 (+  3553us) service_pool.cc:224] Handling call
0709 12:24:24.372167 (+   181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.382896 (+ 10729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.382923 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.382964 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:24.382999 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.657800 (+1274801us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b54000c6054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:25.657847 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.657987 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24285952}
W20260709 12:24:25.687553 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.688006 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:25.688555 31063 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.659924 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:25.690630 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:25.661255 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1829) took 1410 ms. Trace:
I20260709 12:24:25.692515 30989 rpcz_store.cc:276] 0709 12:24:24.250595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.261086 (+ 10491us) service_pool.cc:224] Handling call
0709 12:24:25.661034 (+1399948us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400226854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:25.661189 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1575618816}
I20260709 12:24:25.663160 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 647) took 1522 ms. Trace:
I20260709 12:24:25.663909 30983 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:24:25.701407 30983 pending_rounds.cc:85] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 6
I20260709 12:24:25.701644 30983 pending_rounds.cc:107] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:24:25.701874 30983 raft_consensus.cc:2889] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:25.654390   709 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.705476 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:25.708446 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:25.709450 31068 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:25.710599 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:25.715435 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:25.715978 31068 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:25.710371 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:25.721067 31697 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:25.724856 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 328474us
W20260709 12:24:25.722101   996 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:24:25.728657 31106 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.707729   827 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.685583   823 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.732729 31142 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:25.722631   845 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:25.733911 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 144918us
W20260709 12:24:25.728622   952 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.008s
I20260709 12:24:25.734975 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.724941 31063 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:25.736917 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.739464 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:25.739954 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:25.741037   939 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.740159 31063 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:25.744606 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:25.747100 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:25.747534 31063 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.734935   999 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:24:25.748797  1000 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:25.749266  1000 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.749588 31690 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:24:25.751374   963 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.006s
I20260709 12:24:25.751823 31136 raft_consensus.cc:606] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:25.752169 31136 raft_consensus.cc:614] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:25.755806 31068 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:25.756055 31067 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:25.757227 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:25.734565   845 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:25.743381   814 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:25.701238 31689 rpcz_store.cc:276] 0709 12:24:24.140421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.146513 (+  6092us) service_pool.cc:224] Handling call
0709 12:24:24.171154 (+ 24641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.152594 (+981440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.662902 (+510308us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b5400690354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:25.663092 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2253046016}
I20260709 12:24:25.772168 31063 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:25.780113 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:25.755026   969 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.006s
I20260709 12:24:25.736159  1003 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:25.782003 31063 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:25.773746   769 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:25.795420 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.797374 31067 pending_rounds.cc:85] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 1
I20260709 12:24:25.749610  1000 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.786957   709 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:25.790748   841 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:25.818926   841 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:25.781365   814 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:25.819823   841 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.820053   814 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.791242   823 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:25.822404  1002 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:25.781384   992 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:25.794303  1003 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.823547  1003 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:25.823899 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1901) took 1271 ms. Trace:
I20260709 12:24:25.797266 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 1897) took 1245 ms. Trace:
I20260709 12:24:25.824846 31059 rpcz_store.cc:276] 0709 12:24:24.551734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.950374 (+398640us) service_pool.cc:224] Handling call
0709 12:24:25.797039 (+846665us) spinlock_profiling.cc:243] Waited 92.4 ms on lock 0x7b54000c6f54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:25.797187 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212483456}
I20260709 12:24:25.826144 31059 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:25.828938 31059 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:25.797784 31067 pending_rounds.cc:107] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:25.829628 31067 raft_consensus.cc:2889] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:25.798077 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 373) took 1475 ms (client timeout 1891 ms). Trace:
I20260709 12:24:25.776921   845 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.809517   896 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:25.796778   969 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.772527   930 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:25.824383 31065 rpcz_store.cc:276] 0709 12:24:24.552943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.997544 (+444601us) service_pool.cc:224] Handling call
0709 12:24:25.048832 (+ 51288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.213398 (+164566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.823697 (+610299us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54002ce2d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:25.823843 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376459520}
W20260709 12:24:25.830152 30987 rpcz_store.cc:269] 0709 12:24:24.322518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.322699 (+   181us) service_pool.cc:224] Handling call
0709 12:24:25.797935 (+1475236us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:25.798021 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165169920}
W20260709 12:24:25.805845   953 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.006s
I20260709 12:24:25.834019 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:25.782651 31068 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:24:25.839900 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 103070us
W20260709 12:24:25.843502 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 108598us
I20260709 12:24:25.846975 31059 raft_consensus.cc:2376] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6. Re-sending same reply.
I20260709 12:24:25.847744 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:25.848369 31106 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.851687 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:25.856045 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.838366   822 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:24:25.855598   953 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:25.859272   953 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:25.859666   896 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.860262   896 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:25.863003   818 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:25.848229   930 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:25.849081   909 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
W20260709 12:24:25.870941  1004 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:24:25.878590   989 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:25.878973   989 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:25.858166   864 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:24:25.880362   864 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:25.880764   864 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:25.880858   969 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:25.883054 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:25.883754 31059 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.881701 31061 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:25.885721 31068 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:25.865448   600 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:24:25.903604 31068 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:25.905186   823 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:25.906834   855 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.908020 31059 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:25.908581   994 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:25.908950   855 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:25.909070 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:25.909955   823 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:25.910451   823 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:25.894555   989 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:25.911242   984 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.508s	user 0.003s	sys 0.000s
W20260709 12:24:25.914341 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 62734.6us
I20260709 12:24:25.915138 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:25.914366 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58156.3us
I20260709 12:24:25.919144 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.939287 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:25.962858   962 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:25.934348   855 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:25.965375   855 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:25.965790   855 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.911418   823 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:25.953599   988 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:25.974366   962 raft_consensus.cc:2764] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:24:25.974504   935 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.977360 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.980823 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:25.987133 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.991360 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:25.999462 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.908973   994 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:25.934847   927 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.933687   926 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.897295   969 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:24:26.005851 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:24:26.050421 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 44642us
I20260709 12:24:26.053388 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:25.981808   730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.013s	sys 0.010s
I20260709 12:24:26.077409 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:26.081085 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:25.881645   864 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.081943   970 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:26.084846  1006 raft_consensus.cc:2764] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:24:26.085289  1002 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:24:26.085887   911 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.984942   988 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:24:26.078631   984 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:24:26.095203 31109 leader_election.cc:400] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
I20260709 12:24:26.095521 31109 leader_election.cc:403] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:26.096968 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:26.004184   962 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:26.080390   730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.029s	sys 0.024s
I20260709 12:24:25.911650   930 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.081979   953 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:26.082432   992 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:25.975026   935 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.079658   926 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.083372  1007 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:26.094431   970 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:26.107827   963 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:26.113359   864 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:26.089421   917 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.010s	sys 0.001s
I20260709 12:24:26.081298   927 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.140396   927 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.108162   999 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
W20260709 12:24:26.088557 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 234927us
I20260709 12:24:26.106813  1000 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.124436   952 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:26.098515  1003 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:26.143791   917 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.010s	sys 0.001s
I20260709 12:24:26.101348   911 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.266371 31058 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:26.124660  1006 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:24:26.126081   994 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.215917 31661 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:26.307114 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 126218us
I20260709 12:24:26.295367   953 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:26.218801   848 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 12:24:26.309834   762 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:26.218524   970 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.311991  1011 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.331111   935 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.333212  1003 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.334079   926 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.322929   954 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.338199   953 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:26.338806  1014 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:24:26.328260   994 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:26.340862   980 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.881s	user 0.000s	sys 0.002s
I20260709 12:24:26.345835   955 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.400218  1000 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:26.359118  1011 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:26.402683  1015 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
I20260709 12:24:26.405225  1011 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:26.432752 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:26.436537 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.450274 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.451467 31060 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:26.457981 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1611) took 1075 ms. Trace:
I20260709 12:24:26.458487 31063 rpcz_store.cc:276] 0709 12:24:25.382961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.782724 (+399763us) service_pool.cc:224] Handling call
0709 12:24:25.823113 (+ 40389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.823774 (+   661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.823803 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:25.824308 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:24:25.976461 (+152153us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:25.976973 (+   512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.977151 (+   178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.457543 (+480392us) raft_consensus.cc:1685] finished
0709 12:24:26.457575 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.457818 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":39040,"thread_start_us":151048,"threads_started":1,"wal-append.queue_time_us":306165}
I20260709 12:24:26.467662 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:26.469214 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1502) took 1666 ms. Trace:
I20260709 12:24:26.341027   622 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.469769 31144 rpcz_store.cc:276] 0709 12:24:24.802092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.119209 (+317117us) service_pool.cc:224] Handling call
0709 12:24:25.134902 (+ 15693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.151039 (+ 16137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.151070 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:25.151450 (+   380us) log.cc:844] Serialized 26 byte log entry
0709 12:24:25.356199 (+204749us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:25.409456 (+ 53257us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.484538 (+ 75082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.484629 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.340299 (+855670us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:26.468428 (+128129us) raft_consensus.cc:1685] finished
0709 12:24:26.468608 (+   180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.468953 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15783,"prepare.queue_time_us":59061,"prepare.run_cpu_time_us":16,"prepare.run_wall_time_us":16,"raft.queue_time_us":169328,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":1798,"spinlock_wait_cycles":6238080,"thread_start_us":320541,"threads_started":3,"wal-append.queue_time_us":237514}
I20260709 12:24:26.472663   729 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:26.473404   729 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.473853   729 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:26.474056   814 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:26.475219   814 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.317397   752 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.479811   730 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.480448 31065 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:26.339155   762 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:26.481084   730 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.483302 31065 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:26.484737   730 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.485431 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.487094   762 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.488797 31063 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:26.495647   814 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.337558   845 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.505188   822 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:24:26.505647   752 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.528695   752 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.531564   752 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:26.545055   855 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.001s
I20260709 12:24:26.527523   884 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:24:26.578420   841 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:24:26.582463   955 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.583316   955 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.587486   911 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.596252   926 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.597072   926 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:26.599797 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.585955   955 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.562247   855 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.622409   627 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:26.623631   627 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.614184   955 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.644651   845 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.645867   955 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.646430   911 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:26.647269   911 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.648746   955 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.649497   884 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:26.650317   884 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.651536 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.652828   884 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:26.651566 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:26.655417 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.656008 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:51358 (request call id 23) took 1191 ms. Trace:
I20260709 12:24:26.656575  1011 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:26.656488 31136 rpcz_store.cc:276] 0709 12:24:25.464066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.640633 (+176567us) service_pool.cc:224] Handling call
0709 12:24:26.655519 (+1014886us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400063454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88e853 00007f2ef085c60a 00007f2ee9de27d4 00007f2ee9de2762 00007f2ee9de2691 00007f2ee9de260c 00007f2ee9de18c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:26.655711 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254200576}
I20260709 12:24:26.657316 30958 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:26.664767 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.025s	sys 0.037s
W20260709 12:24:26.665177 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.025s	sys 0.037s
I20260709 12:24:26.653894 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 493) took 1322 ms. Trace:
I20260709 12:24:26.675278   926 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.671799 31061 rpcz_store.cc:276] 0709 12:24:25.331524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.337439 (+  5915us) service_pool.cc:224] Handling call
0709 12:24:26.653447 (+1316008us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b54001bb554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:26.653678 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248841600}
I20260709 12:24:26.650041   911 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.657608  1011 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.691365   627 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:26.694555 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 597588us
I20260709 12:24:26.680342   884 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:26.700444   855 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:26.701454 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 747446us
I20260709 12:24:26.703799 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:26.704339   600 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:26.704923   600 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:26.707227   970 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:26.711649  1011 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:26.713084   729 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.711443   600 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.715198   600 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:26.717633   952 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:26.687709   981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.209s	user 0.000s	sys 0.034s
I20260709 12:24:26.723958   952 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:26.756438   823 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.757311   823 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.759564   729 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:26.748198   900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.215s	user 0.012s	sys 0.009s
W20260709 12:24:26.760783   900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.358s	user 0.012s	sys 0.009s
W20260709 12:24:26.762270 30958 connection.cc:256] RPC call timeout handler was delayed by -1.1353693970004315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
I20260709 12:24:26.764268   969 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:26.765100   969 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.766544  1027 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:26.767730   969 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.768630   884 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:26.769258   969 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.770010   969 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.770637   842 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:26.771386   842 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.772431   969 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.777899 30987 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:24:26.778308 30987 pending_rounds.cc:85] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 5
I20260709 12:24:26.778595 30987 pending_rounds.cc:107] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:24:26.778916 30987 raft_consensus.cc:2889] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:26.779331 30986 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:24:26.779944  1013 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.474s	user 0.001s	sys 0.000s
I20260709 12:24:26.781275   841 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:26.782049   841 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:26.783855 30988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1836) took 2020 ms (client timeout 2357 ms). Trace:
I20260709 12:24:26.784526   841 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:26.784328 30988 rpcz_store.cc:269] 0709 12:24:24.763040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.763294 (+   254us) service_pool.cc:224] Handling call
0709 12:24:26.783376 (+2020082us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b54000244d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:26.783641 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914286464}
I20260709 12:24:26.785180   841 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.765864   952 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.788699   952 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.789836 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:26.758041 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.790536 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1908) took 2146 ms. Trace:
I20260709 12:24:26.791042 31058 rpcz_store.cc:276] 0709 12:24:24.644066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.097794 (+453728us) service_pool.cc:224] Handling call
0709 12:24:25.108016 (+ 10222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.306164 (+1198148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.789915 (+483751us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400383954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:26.790299 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2663460096}
I20260709 12:24:26.791632 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:26.791952 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:26.792306 31060 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:26.792533 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:26.793337 31058 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.793862 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:26.794989 31691 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:26.794871 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:26.759677   823 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:26.754643 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48420.9us
W20260709 12:24:26.752611   981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.348s	user 0.000s	sys 0.035s
I20260709 12:24:26.795274 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.773873   842 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.769498   884 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.785725   841 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:26.778028  1012 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.557s	user 0.003s	sys 0.000s
W20260709 12:24:26.817751 31064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 1912) took 2172 ms (client timeout 1867 ms). Trace:
I20260709 12:24:26.826516 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.827318 31065 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
W20260709 12:24:26.828102 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:26.828816 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.829348 31692 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:26.830511   900 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.831322   900 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.820168   709 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:24:26.819846   855 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:24:26.819617  1027 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:24:26.835005 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:26.837407   962 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:26.837884   962 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:26.838809 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:26.839655 30989 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:26.839377   962 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.841116   962 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:26.841921 30984 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:26.842811 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:26.843212 30991 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:26.843617 30985 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
W20260709 12:24:26.844060  1018 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.535s	user 0.000s	sys 0.001s
I20260709 12:24:26.844956 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:26.845831 30983 raft_consensus.cc:2410] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:26.846727 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:26.820176 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.846959   884 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.791975 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:26.848563 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56929.7us
I20260709 12:24:26.849568 31033 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:26.849985   913 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:26.850358   939 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.850970 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1604) took 1476 ms. Trace:
I20260709 12:24:26.851282 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.852064 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:26.852530 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.852696 31694 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.852912 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:26.855808 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:26.856101  1002 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:26.856786 30988 raft_consensus.cc:2410] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:26.850943   939 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:26.818186 31064 rpcz_store.cc:269] 0709 12:24:24.645240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.868443 (+223203us) service_pool.cc:224] Handling call
0709 12:24:26.817376 (+1948933us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54001a6dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:26.817569 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763258240}
W20260709 12:24:26.860073 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.828236 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.867188 31058 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:26.870584 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.856926  1000 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:26.873716 31061 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:26.877003   855 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:26.877621   855 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:26.878961   855 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:26.879875   841 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.880409   855 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.881511  1000 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.882041  1000 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.882352   709 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:26.882830   709 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:26.884007 31696 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:26.884054   709 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.885924   709 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.886895   970 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:26.906522 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 656) took 1521 ms. Trace:
I20260709 12:24:26.906687   939 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:26.907011 31697 rpcz_store.cc:276] 0709 12:24:25.385427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.387482 (+  2055us) service_pool.cc:224] Handling call
0709 12:24:25.387739 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.721484 (+333745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.890002 (+1168518us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b540076ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:26.906306 (+ 16304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762029952}
W20260709 12:24:26.908120   994 consensus_meta.cc:287] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.009s
I20260709 12:24:26.851501 31067 rpcz_store.cc:276] 0709 12:24:25.374467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.757160 (+382693us) service_pool.cc:224] Handling call
0709 12:24:25.797170 (+ 40010us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:25.844050 (+ 46880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.844083 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:25.844592 (+   509us) log.cc:844] Serialized 52 byte log entry
0709 12:24:25.982157 (+137565us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:26.267389 (+285232us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.306907 (+ 39518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.306975 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.716787 (+409812us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b18002c2a80. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a7522 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:24:26.777174 (+ 60387us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:26.849571 (+ 72397us) raft_consensus.cc:1685] finished
0709 12:24:26.849591 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.850651 (+  1060us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54000c6f50. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:26.850790 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39254,"prepare.queue_time_us":342018,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":41976,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":4859,"spinlock_wait_cycles":161987328,"thread_start_us":312401,"threads_started":2,"wal-append.queue_time_us":288580}
I20260709 12:24:26.909107 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:26.914232 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:26.915676 31065 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:26.918926  1030 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:26.919118 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:26.919427  1030 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:26.920076 31068 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:26.920279 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:26.920895 31058 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.920535  1030 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.921963 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:26.923202 31064 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:26.888804 30990 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 12:24:26.860962  1017 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.470s	user 0.001s	sys 0.000s
W20260709 12:24:26.898362 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48998.2us
I20260709 12:24:26.881847   989 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:26.837396   900 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.902999 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:26.903244 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:26.927754 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.903940 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:26.904273 30983 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:26.900866  1039 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.054s	user 0.000s	sys 0.002s
I20260709 12:24:26.902602 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:26.925822 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 64394.5us
I20260709 12:24:26.931309 30986 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:26.933674 31060 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:26.937143 31068 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:26.936102 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:26.945183 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:26.927343   913 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.953769 31141 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:26.954200 31141 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:26.955227 31141 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.956838 30988 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.958910 31064 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:26.960307 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:26.960879 31136 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:26.962364 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:26.963363 31068 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: [5.5-7.6]
I20260709 12:24:26.963903 31067 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:26.964902 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:26.968201 31694 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:26.969288 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:26.970072 31696 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:26.971306 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:26.972925 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:26.975581   864 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:26.962406   913 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:26.980232 30955 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:26.980835 31065 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:26.982379 31058 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:26.984856 30988 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:26.950558   814 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.981932 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:27.007222 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.008013 31136 raft_consensus.cc:2410] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:26.974155   994 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:27.007256 31061 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:27.010154 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:27.004274 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:27.002063   841 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:27.002432   864 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:26.990775   913 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:26.996722   969 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:27.003906   955 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:27.013692 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:26.999696   814 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.009104   930 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.993911  1002 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:24:27.012004 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:27.002274   900 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:24:27.024432 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:27.029280 31697 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:27.031590 31064 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:27.032800   953 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:27.033179   622 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:27.033545   991 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.075251 31063 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:27.033586   823 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.035305   729 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:27.036115 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:27.037081   884 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:27.038162  1027 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.039871 31137 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:27.027830   864 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:27.053578 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.079523 31143 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.027009  1030 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:27.019573   841 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:27.017843   709 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
W20260709 12:24:27.082263 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59276.3us
I20260709 12:24:27.042291 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:27.071973   913 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:27.069476   953 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:27.099565 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 86963.9us
I20260709 12:24:27.074018  1002 consensus_queue.cc:1243] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:24:27.095183 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 74140.1us
W20260709 12:24:27.092270 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 56228.9us
I20260709 12:24:27.026351   994 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.020514  1000 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.094296 31692 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:27.090174   762 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.091079   841 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.110625 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.111243   975 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:27.111647   975 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:27.111997   975 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.112267 31064 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:27.079945   955 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.107924 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1900) took 2450 ms. Trace:
I20260709 12:24:27.116708 31141 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.118824 30992 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:27.119314   884 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:27.119817   884 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:27.120925   884 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.100862   953 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.103049   709 consensus_queue.cc:1243] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:27.085562   969 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.129815  1000 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:27.130097 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.131111   919 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.129815 31063 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:27.138389 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:27.145794 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 163885us
I20260709 12:24:27.148773 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.149606 31139 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:27.154157 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.154904 31137 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:27.156097 31138 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:27.163565   994 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:27.115834  1027 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.117451 31062 rpcz_store.cc:276] 0709 12:24:24.552655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.994865 (+442210us) service_pool.cc:224] Handling call
0709 12:24:24.995628 (+   763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.995833 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:26.794823 (+1798990us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18001d5360. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf
0709 12:24:26.795034 (+   211us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.942672 (+147638us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.952056 (+  9384us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:26.952121 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.997158 (+ 45037us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.003063 (+  5905us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45858,"prepare.queue_time_us":66786,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":111,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4845,"spinlock_wait_cycles":7423360,"thread_start_us":178270,"threads_started":2,"wal-append.queue_time_us":218169}
I20260709 12:24:27.101116   622 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.090689   930 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.181874 31136 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:27.182057 30986 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:27.100138   991 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.183840 31140 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:24:27.101115   823 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.126027   762 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:27.273389 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 107028us
W20260709 12:24:27.247076 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45966.8us
I20260709 12:24:27.184906 31142 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
W20260709 12:24:27.286067 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 176821us
W20260709 12:24:27.262753  1015 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:24:27.310956 31143 raft_consensus.cc:2410] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:27.313205   969 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.273743 31140 pending_rounds.cc:85] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 4
W20260709 12:24:27.322643  1051 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 12:24:27.273738   842 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:24:27.325461 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:27.271590  1050 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:24:27.313594   841 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.272060  1027 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:27.329218 31140 pending_rounds.cc:107] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:24:27.219090   939 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.329828   952 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.004s
I20260709 12:24:27.349591   930 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.350495 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:27.350623  1056 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.350955 31137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:27.351759 31137 raft_consensus.cc:606] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:27.351711  1056 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:27.353221   823 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.354254   841 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.354817   991 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:27.308394  1052 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:24:27.181453   919 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.346597   814 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.339483 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.330991 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:27.368451   955 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.338308   962 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:24:27.370169   962 consensus_queue.cc:1243] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:27.370662   970 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:24:27.343282 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.272828  1000 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 12:24:27.345173 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.371555 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.338526 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:27.368114   762 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.219600   864 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.341333 31140 raft_consensus.cc:2889] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:27.347323   939 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.377933 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:27.379916 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.381208 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 106110us
I20260709 12:24:27.371126   970 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.384130 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:27.385576 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50419.7us
I20260709 12:24:27.393438 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.394245 31144 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:27.395076 31138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:27.395727 31138 raft_consensus.cc:606] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:27.395650   970 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.397106   970 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:27.397200   622 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.398619 31141 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:24:27.398960 31141 pending_rounds.cc:85] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 4
I20260709 12:24:27.399204 31141 pending_rounds.cc:107] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:24:27.399435 31141 raft_consensus.cc:2889] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:27.399819   939 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.400609 31139 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:24:27.401747   600 consensus_meta.cc:287] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.000s
W20260709 12:24:27.406296 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.405630   975 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.408648   975 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.384183   930 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 12:24:27.410455   987 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:24:27.411397   919 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.413872 30988 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:27.414253 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:27.414906 30983 raft_consensus.cc:2376] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5. Re-sending same reply.
I20260709 12:24:27.415122 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.415970 30990 raft_consensus.cc:2410] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:24:27.416162 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90853.7us
I20260709 12:24:27.394320 31062 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:27.418532 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:27.418895 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:27.419584  1000 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:27.420334  1000 consensus_queue.cc:1243] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:27.420780  1000 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:27.421123  1000 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.394522 31033 leader_election.cc:400] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4.
I20260709 12:24:27.422483  1002 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:27.424245 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.428277   953 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.434998 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:27.437764 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 98366.8us
I20260709 12:24:27.439947 31141 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:27.440064 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.440078 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.458459 31137 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:27.441159 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.443034  1023 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:27.417272   855 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:24:27.418068   952 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:27.445710 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:27.446696 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:27.447098 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.449833 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.450510 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.451051 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:27.451551   987 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:27.421958  1000 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:27.452512   989 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.521s	user 0.006s	sys 0.003s
I20260709 12:24:27.440089 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.460140 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.464965 30990 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:27.462146 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.465929 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:27.468091 30989 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:27.467087 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.468118 31059 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:27.467429 31135 raft_consensus.cc:2410] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:27.461370   622 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:27.470448 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.471064 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:27.473342 30983 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:27.473455 30991 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:27.475901 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.477274 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:27.463012  1023 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.473909 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.478662 31067 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:27.477643   952 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.482816   952 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.485410 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.486584 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.491534 31144 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:27.492995  1059 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.493678 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.493575  1059 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.494316 31059 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:27.491560   622 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.495779  1059 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.491201 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:27.496729 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.497642 31060 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:27.498674 31106 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.499369 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.472049   987 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.500003 31141 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:27.500891 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.500633  1023 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.500764   987 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.480585 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:27.496771   622 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.493928 31139 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:27.472330  1046 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:27.505191 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:27.501746  1023 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.509757   975 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:27.514288 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 67765.3us
W20260709 12:24:27.515316 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 91181.5us
I20260709 12:24:27.518105 31106 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:27.519052   991 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:27.522200 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:27.525090 31144 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:27.526530 31141 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:27.528761   953 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:27.511603  1002 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.559139  1000 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:27.570616  1066 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:24:27.577127 31139 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:27.524505   935 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:27.580008 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61990us
I20260709 12:24:27.568729   991 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:27.582683 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.530963   855 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:27.585450 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 63317.9us
I20260709 12:24:27.587867 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:27.568529   953 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:27.594928 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.596911 31033 leader_election.cc:400] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:27.597165 31033 leader_election.cc:403] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:24:27.588779  1030 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:24:27.599746   762 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:27.599924 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:27.600694   901 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:27.601069   901 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:27.601987   901 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.602497 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:27.603253   901 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.603560 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:27.604300   989 raft_consensus.cc:2764] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:27.604933   989 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:27.605257   989 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.605569   989 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:27.606535   864 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:27.610621 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.585988   935 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.617146   913 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:27.622575 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:27.623976 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:27.624485 31059 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.626487 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.627610 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.628927 31060 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:27.629447 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 94917.3us
I20260709 12:24:27.632035 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:27.613926   622 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.634225 30991 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.614104  1027 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:27.621860   935 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.630709   953 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:24:27.640727 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 43937us
I20260709 12:24:27.636297   709 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:24:27.633965   901 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:24:27.637318  1027 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.633285   762 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:24:27.651279   709 consensus_queue.cc:1243] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:27.661799 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:27.662398 31136 raft_consensus.cc:606] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:27.662741 31136 raft_consensus.cc:614] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:27.664047 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:27.667709 30989 raft_consensus.cc:1217] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Deduplicated request from leader. Original: 4.3->[6.4-6.4]   Dedup: 6.4->[]
I20260709 12:24:27.668203 30989 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:27.670681 31692 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:27.671964 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:27.673455   769 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.676823 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.682987 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.683688 31140 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:27.683740 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.684222 31139 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:27.684408 31136 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:27.682329 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.673983   769 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.695920 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.696506 30986 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:27.699332   627 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:27.699462   981 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:27.699929   981 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.700781   981 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.694608 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:27.703895 31138 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:27.706276   769 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.706593   953 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.666957  1030 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:27.715580  1030 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.717550  1030 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:27.719347 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:27.720122 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 68490.9us
I20260709 12:24:27.720913 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.722394   627 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:27.727782   901 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:27.728183   901 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:27.730846   962 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:27.731215   962 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.732009   962 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.733572  1002 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:27.740082 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:27.729979   901 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.742795   901 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.743731  1027 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.747800  1023 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:27.748319  1023 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.713603 31059 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:27.714999  1065 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:27.649549   987 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:24:27.749888 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:27.790807 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:27.791536 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 69461.2us
I20260709 12:24:27.706207 30991 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:27.750973 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:27.757113 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.867478 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:27.882189   981 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.863343   989 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:27.870985 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 173351us
I20260709 12:24:27.921077 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.921790 31060 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:27.924253 31064 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:27.924212   822 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:27.924558   953 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.826367 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 218508us
I20260709 12:24:27.925204   953 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.925529 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.927335   953 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:27.928256 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.928774  1023 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.929219  1002 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.929665  1002 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.930963 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.931725  1002 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:27.931916  1059 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.933072 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:27.933383  1059 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:27.879540  1030 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:27.934784  1030 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.935792  1030 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.870734   935 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.936970  1030 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:27.938500 30990 raft_consensus.cc:2368] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6.
I20260709 12:24:27.865528 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.940256  1065 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.749626 30889 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:27.838491  1054 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:27.855161 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 104680us
I20260709 12:24:27.790315   939 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:27.895969   975 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:27.941982   864 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:27.941903 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:27.951862 31060 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.943148   919 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:27.943287 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.860064   962 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:27.941228 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75799.5us
W20260709 12:24:27.946419 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 195505us
I20260709 12:24:27.950681   953 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:27.957464 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.959295 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.960556 31063 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:27.960945 31063 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:27.961408 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.962558 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.962716 31068 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:27.960757 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:27.963515  1002 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:27.963840 31065 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:27.964398   953 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:27.961719 31063 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.965432   953 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:27.966199 31063 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:27.966782 30888 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 317) took 10160 ms. Trace:
I20260709 12:24:27.967309 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:27.968624  1056 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:27.969321  1056 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:27.970791 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:27.947763   989 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.945604   935 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:27.956446   919 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:27.975579   919 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.972002 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:27.971591  1056 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:27.989225 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:27.951274   975 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:27.998298 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:27.998067 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:27.972419  1023 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:28.001471 30986 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:27.979964  1059 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.016510 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.018150 30984 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:28.024326 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:27.990723  1056 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.027742  1056 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.028335 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:28.029223   627 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:28.029657 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:28.030653   991 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.030746   627 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:28.031220   991 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.033289   991 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:28.033784   954 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.035480   975 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.002249   901 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.026660  1059 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.036239   901 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.037875  1059 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.001914   709 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:28.046591   884 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.047019   884 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.047093   935 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:28.047423   884 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.047919  1065 raft_consensus.cc:2764] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:28.049513  1054 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.050338   919 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.050002  1054 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.055336  1054 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:27.967183 30888 rpcz_store.cc:276] 0709 12:24:17.848010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.874267 (+ 26257us) service_pool.cc:224] Handling call
0709 12:24:18.970517 (+1096250us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fe090 00007f2eeb89cf3a 00007f2ef1652e9c 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:27.966538 (+8996021us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:23.885189 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:24.012384 (+127195us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:24.020159 (+  7775us) write_op.cc:435] Acquired schema lock
0709 12:24:24.022325 (+  2166us) tablet.cc:662] Decoding operations
0709 12:24:24.210241 (+187916us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:24.210333 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:24.210362 (+    29us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:24:24.211482 (+  1120us) tablet.cc:701] Row locks acquired
0709 12:24:24.211501 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:24:24.211669 (+   168us) write_op.cc:270] Start()
0709 12:24:24.211773 (+   104us) write_op.cc:276] Timestamp: P: 1783599864211621 usec, L: 0
0709 12:24:24.211800 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:24:24.212458 (+   658us) log.cc:844] Serialized 5192 byte log entry
0709 12:24:24.982771 (+770313us) op_driver.cc:464] REPLICATION: finished
0709 12:24:25.131736 (+148965us) write_op.cc:301] APPLY: starting
0709 12:24:25.131853 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:25.132208 (+   355us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:25.132345 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:25.249015 (+116670us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:25.249262 (+   247us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:24:25.249306 (+    44us) write_op.cc:312] APPLY: finished
0709 12:24:25.249599 (+   293us) log.cc:844] Serialized 95 byte log entry
0709 12:24:25.249791 (+   192us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:25.249929 (+   138us) write_op.cc:454] Released schema lock
0709 12:24:25.249983 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:25.250384 (+   401us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:25.250416 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":56482,"reported_tablets":32,"spinlock_wait_cycles":2472044928,"child_traces":[["op",{"apply.queue_time_us":145028,"apply.run_cpu_time_us":7678,"apply.run_wall_time_us":120327,"num_ops":13,"prepare.queue_time_us":316599,"prepare.run_cpu_time_us":19861,"prepare.run_wall_time_us":502546,"raft.queue_time_us":529481,"raft.run_cpu_time_us":15518,"raft.run_wall_time_us":198288,"replication_time_us":752362,"thread_start_us":895029,"threads_started":4,"wal-append.queue_time_us":155828,"wal-append.run_cpu_time_us":35132,"wal-append.run_wall_time_us":1899537}]]}
I20260709 12:24:28.057391 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.058260 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:28.062039 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.066249 32721 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:28.008528  1000 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:28.067328 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.066957 30889 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.063169 31062 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:28.068105 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:28.066030 31060 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:28.071857 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:27.988893   600 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:24:28.034178   954 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.038163   901 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:28.078514 31063 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:28.079051 30983 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:28.082582 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.085305 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:28.069818  1000 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.094192 30991 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:28.072794  1027 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.079226 30889 catalog_manager.cc:5697] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:28.096005 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.074652   939 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:28.083303   954 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:28.099642 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73988.5us
I20260709 12:24:28.095676   970 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:28.094080   709 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:28.116882   709 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.095924   975 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:24:28.100234 30889 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 3 to 4, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.101282   991 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:28.101559 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.102012 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:28.112850   952 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:28.114554 31062 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:28.083554  1054 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:28.096611  1000 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.122552 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:28.118847  1027 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.129726 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:28.130013   970 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:24:28.131392   970 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:28.121907  1002 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:24:28.134776 31060 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:28.136919 31060 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:28.121218   991 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.128058   935 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:28.134675 30889 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 3 to 4, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.137873 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.139786 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.144233 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.144976 31065 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:28.146051 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 200734us
I20260709 12:24:28.137452   969 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:28.149024 30983 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:28.153862 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:28.142509   991 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.149958 31060 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.143603   935 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.162140 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:28.152602 32721 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:24:28.163385   969 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:28.172794 32721 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:28.173199 32721 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:28.173405   969 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.174008 32721 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.175423 32721 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:28.138336   952 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.163671   935 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:28.182792 30889 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 3 to 4, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 4 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:28.165534  1074 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:28.164954  1061 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:24:28.219899 31063 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:28.237545   864 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:28.186843  1056 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:28.238543 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 94464.7us
W20260709 12:24:28.238620 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 295381us
I20260709 12:24:28.223703  1052 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:28.247896 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:28.251013 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.256614 31065 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:28.258324 30889 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 4, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 4 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:28.250085  1052 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:28.264284  1075 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:24:28.264034   901 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:24:28.265678  1078 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:24:28.252912   729 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:24:28.258818   822 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
W20260709 12:24:28.272825 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.269783  1052 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:28.278697  1084 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:24:28.273026 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.282167 31697 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:28.283274 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:28.284439 31692 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:28.284611 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:28.286901  1090 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.287304  1090 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.287446 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.287696  1090 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.288275 31698 raft_consensus.cc:2410] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:28.289297 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.289661 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.290004 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.290287 31141 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.290553 31137 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:28.290627 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.291720 31139 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:24:28.291913 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.292477 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:28.292788 31138 raft_consensus.cc:2410] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:28.293620 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.294181 31065 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:28.294257   901 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:28.294500 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.294754   901 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:28.295148 31140 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:28.289661 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.296149 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:28.298195  1088 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:28.289326 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.298763  1088 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.299182  1088 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.286899 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:28.299177 31143 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 3: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:28.301015 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.275267  1056 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.302538 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:28.284821   762 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:28.303342 31697 raft_consensus.cc:2410] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:28.295186   901 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.290830 31136 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:28.306452  1088 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.305648   864 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.310459   962 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.310946   962 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.311789 31135 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:28.312948   962 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.314849   952 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.314566   884 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.316723   884 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.313613   954 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.271014  1074 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:24:28.318471  1088 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.318794   954 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.319573   935 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.320904 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:28.321110   709 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:28.325392 31664 leader_election.cc:400] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6.
I20260709 12:24:28.325717 31664 leader_election.cc:403] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:28.325744 30991 raft_consensus.cc:2368] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9.
I20260709 12:24:28.327709  1088 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.327911 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.328198  1088 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.329387   935 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.327955  1052 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.323173  1090 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.344043 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.330416 31136 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:28.274520  1081 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:24:28.316133 31137 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:28.332453 31135 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:28.332890 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:28.273825  1082 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:28.334017 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:28.337764 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.330232   901 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.339066 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.398587 31143 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.345837 31141 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:28.336566 30889 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.401800 30889 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 4 to 7, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 7 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.390430 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:28.403666 30991 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.346828 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:28.393020 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 60201.8us
I20260709 12:24:28.347882 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.404449 31063 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.404757 31109 leader_election.cc:400] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9.
I20260709 12:24:28.404675 31142 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:28.405162 31109 leader_election.cc:403] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:28.347334 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.337127  1088 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.340848   989 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.409662 31695 raft_consensus.cc:2368] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
W20260709 12:24:28.410777 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 162148us
I20260709 12:24:28.412351 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:28.413298 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:28.413859 31065 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:28.415156 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.416173 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.418008 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.391180   769 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.414624 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.419725 31136 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:28.420183 31033 connection.cc:256] RPC call timeout handler was delayed by -1.5170571520002341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23570000000000002s for a response
W20260709 12:24:28.420838 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 86929.8us
I20260709 12:24:28.418292 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:28.424273 30989 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.421168 30889 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:28.402736   953 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:28.427909 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.428517 31139 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:28.430444 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.431337 31143 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:28.407089   939 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:28.432885 31140 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:28.412508   975 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:28.431105 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:28.411159   864 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.435719 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.438166  1089 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:28.446520 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.446693  1089 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.449106  1089 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:28.456419 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.457335 31694 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:28.440080 31062 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:28.476588 31063 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:28.442797 30991 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:28.469820 30989 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:28.418910   952 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:28.444391  1092 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:28.443907 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.487499 31689 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:28.488998 31064 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:28.492007   962 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:28.492579   962 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.426079   627 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:28.494580   627 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:28.484380 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 56860us
I20260709 12:24:28.444240 31692 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:28.511421   627 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.480471  1052 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.435861  1059 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.432366   769 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.512343  1059 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.512472 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.512768  1059 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.443531  1027 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:28.418625   981 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:24:28.432256   953 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.444264 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.514431   953 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.484916 31058 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:28.516093 31058 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:28.517164 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.517741 31697 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:28.513926   769 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.520220 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.527868 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.521687 30889 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 4 to 5, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 5 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.531315 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:28.535504 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.537797 31060 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.538681 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:28.539215 30990 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:28.541030 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.541523 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.445607 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.535766 31058 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.549118 31058 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:28.549228 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 68567us
I20260709 12:24:28.554759 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.555929 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.556740 30983 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:24:28.557278 31694 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:24:28.557541 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.558238 30992 raft_consensus.cc:2393] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:28.558136 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:28.558513 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.558908 30984 raft_consensus.cc:2368] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9.
I20260709 12:24:28.558521 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.445253   864 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.558802 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.561048   864 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.561815 31696 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:28.561935 31058 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:24:28.562448 30889 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 4 to 6, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.564266 30889 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 4 to 6, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.565625 30990 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:28.484119   939 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.569052 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:28.565840 30889 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 5 to 6, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.527400  1002 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:28.574275 30889 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.575754 31060 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:28.577924 31143 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:24:28.580745 31143 pending_rounds.cc:85] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 5
I20260709 12:24:28.581065 31143 pending_rounds.cc:107] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:24:28.581369 31143 raft_consensus.cc:2889] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.581406 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.579957 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:28.582262 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.583751 31136 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:28.583856 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.584769 31142 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:28.586632 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.587744   901 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:28.588704  1080 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:28.590787  1090 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.590912 31144 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:28.591325 31144 pending_rounds.cc:85] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 3
I20260709 12:24:28.591562 31144 pending_rounds.cc:107] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:24:28.591785 31144 raft_consensus.cc:2889] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.591954 30991 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:28.590915 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:28.582880 31058 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:28.584059 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.602157 31143 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:24:28.602515 31143 pending_rounds.cc:85] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 2
I20260709 12:24:28.602720 31143 pending_rounds.cc:107] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:24:28.602926 31143 raft_consensus.cc:2889] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.605310 30889 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 2 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:28.607429 31142 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:28.607911 31140 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:24:28.607491 31061 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:28.545536  1027 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:24:28.599606 31691 raft_consensus.cc:2393] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 2: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:28.613772 31661 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:28.614310 30889 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 3 to 4, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 4 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:28.613051 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:28.618206 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:28.620128 31143 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:24:28.622102 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:28.622677 31058 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.620627 31143 pending_rounds.cc:85] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 2
I20260709 12:24:28.625145 31143 pending_rounds.cc:107] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:24:28.584573   970 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:24:28.625464 31143 raft_consensus.cc:2889] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.604782  1056 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:24:28.613190   962 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.627494 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:28.609967   730 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:28.594391  1059 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.642849 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.698599 31696 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:28.627745   939 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.584125  1000 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.596949  1052 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:24:28.679911 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.624135  1094 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:28.680172  1027 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:28.684607   989 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6.
I20260709 12:24:28.741858  1088 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.743922  1088 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:28.685418 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:28.689014 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 70797.9us
W20260709 12:24:28.690404 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.691079   627 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:28.693475 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:28.600075  1002 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:28.812389 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 43948.8us
I20260709 12:24:28.617180  1080 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:28.709723 31065 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:28.628890   901 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:28.738365 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 120233us
I20260709 12:24:28.740722   935 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:28.741695  1056 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:28.783849 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:28.686108   975 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9.
I20260709 12:24:28.768745 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.823475 31141 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.731380   962 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.786273 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.831805 31135 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.817282 31058 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:28.835661 31141 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:28.844290 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.824224   901 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.791385   954 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:24:28.823889 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:28.852512 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.783455  1094 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:24:28.822762   935 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:28.827410  1056 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:28.850634 31136 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:28.828830   627 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:28.863123 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:28.880705 31696 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.875955 31135 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:28.857597   901 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:28.884184 31141 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:28.887467 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:28.888098 31135 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.839285   962 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.901835 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:28.887110   991 consensus_meta.cc:287] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.717s	user 0.010s	sys 0.000s
I20260709 12:24:28.910458 31135 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:28.875716  1065 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 12:24:28.914186  1100 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 12:24:28.915395   991 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.930960  1059 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:28.929842  1000 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.931734  1088 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.824641  1080 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:28.949576   953 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:28.955713  1002 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.963642 31696 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:28.921844   935 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:28.996637 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 160117us
I20260709 12:24:28.997746  1117 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:28.998386  1117 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.001006  1117 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:29.001543  1120 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.002090 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:29.002046  1120 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.002897 31137 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:29.002919  1074 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:29.004133  1120 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:29.004382  1074 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:29.005829  1080 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.968603  1088 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.006481  1074 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:29.008067  1088 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:29.009260   962 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:28.938454   627 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.015816  1030 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.016489  1030 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:29.019035   991 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:29.019460 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:29.020527 31698 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:24:29.024398 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 120621us
I20260709 12:24:29.026629   901 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.028852   901 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:29.030253   953 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:29.036741   901 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.037402   901 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:29.039098 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.039911   901 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:29.041854 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:29.042804   919 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:29.043380   919 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:29.044495   919 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.048422 31058 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:29.049316 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:29.071723 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:29.074301 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:29.075304 31696 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:29.093230 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:29.088434   730 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:29.105072 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:29.107398 31696 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:29.076205   935 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:29.105202 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:29.105252 31137 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:29.106011 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:29.096706  1094 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 12:24:29.121184 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.113698   730 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:29.133837  1119 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:24:29.106441  1074 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:24:29.136250  1074 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:29.136627  1074 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:29.136970  1074 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:29.123878  1116 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:24:29.139520 31698 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:29.141301 31061 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:29.143260 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.143210 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:29.144105 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:29.144289 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:29.142098 31690 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:29.154259 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:29.160097  1061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.016s	sys 0.037s
I20260709 12:24:29.189908   730 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:29.150866  1030 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:29.165652  1129 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:29.182446  1061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.017s	sys 0.037s
I20260709 12:24:29.194281  1054 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:29.202278  1129 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.220924  1129 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:29.276793  1099 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.663s	user 0.002s	sys 0.000s
W20260709 12:24:29.279189 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.220737  1054 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:29.324450 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 219291us
I20260709 12:24:29.314252   919 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:29.354265  1133 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:24:29.414836  1074 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:29.414731  1056 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.433472  1054 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:29.433967  1056 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:29.314643   970 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.319715   901 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.417555 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:29.419368  1137 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 12:24:29.419610  1131 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:24:29.440400 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:29.425707  1129 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.392568  1117 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:29.431625 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:29.443996 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:29.444818 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:29.432876 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:29.440347  1140 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:29.446684 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:29.447366   884 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:29.452018   884 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:29.452338 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:29.454807 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:29.447316 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:29.469246 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:29.435340   919 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.487303 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:29.437594   970 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.447727   935 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:29.487560 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:29.450834  1123 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:29.468937   884 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:29.438194   901 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.441799  1117 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.485893  1142 raft_consensus.cc:2764] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:24:29.499419 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:29.486078  1140 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:29.487852  1052 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.487455   969 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:29.489046  1138 raft_consensus.cc:2764] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:24:29.517946   919 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:29.518020 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:29.518301  1117 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:29.518898   919 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.519484 31690 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:29.520087 31690 pending_rounds.cc:85] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 5
I20260709 12:24:29.519447   919 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.520347 31690 pending_rounds.cc:107] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:29.520633 31690 raft_consensus.cc:2889] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.503337  1065 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:29.507055  1054 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:29.526190 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:29.530578 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:29.529203 31109 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.535s (SENT) [suppressed 40 similar messages]
I20260709 12:24:29.500213  1123 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.522554   969 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:29.522003  1153 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:29.549183 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:29.528632  1054 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:29.546610 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:29.551059  1054 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:29.552932   939 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.554353 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:29.555222   939 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:29.556552 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:29.557458   939 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:29.559137 31033 leader_election.cc:400] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6.
I20260709 12:24:29.561148 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:29.562420  1140 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:29.563580 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:29.565009 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:29.565552 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:29.563685 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:29.530936   769 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:29.569406 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:29.570614 31135 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:29.522872  1065 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:29.573130 31144 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:29.575392  1074 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:29.575997 31692 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:24:29.576152 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:29.581435 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:29.581967 31063 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.580392 31695 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:29.535596  1088 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:29.583379 31137 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 5. Ops: []
I20260709 12:24:29.585325 31143 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 7. Ops: []
I20260709 12:24:29.549857  1155 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.586448 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:29.578413  1153 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:29.531893   864 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:24:29.569020   769 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.579524  1065 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:29.548255  1142 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:24:29.608790  1129 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:29.609324   969 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:29.610736   901 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:29.600693   769 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.611575  1123 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:29.600380 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 71206us
I20260709 12:24:29.585654  1088 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.628091 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:29.644691 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 112017us
I20260709 12:24:29.646440 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:29.647141 31694 raft_consensus.cc:2368] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6.
I20260709 12:24:29.647482 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 675) took 1080 ms. Trace:
I20260709 12:24:29.641021 31063 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:29.647751 31694 rpcz_store.cc:276] 0709 12:24:28.566623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.566728 (+   105us) service_pool.cc:224] Handling call
0709 12:24:29.646298 (+1079570us) spinlock_profiling.cc:243] Waited 7.59 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:29.647430 (+  1132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17457024}
I20260709 12:24:29.651299 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:24:29.644840  1136 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:24:29.639354   919 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:29.645977  1088 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:29.674096 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:29.674940 31063 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:29.594146  1155 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:29.660349   970 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:29.706809 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:29.708330 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:29.709123 31060 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
W20260709 12:24:29.753170 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 178491us
I20260709 12:24:29.755700 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:29.757529 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 104711us
I20260709 12:24:29.759196 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:29.759804 31060 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.682915  1135 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:29.696080  1155 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:29.762534 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 199528us
W20260709 12:24:30.345728 31144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.000s	sys 0.027s
W20260709 12:24:30.347803 31144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.000s	sys 0.028s
I20260709 12:24:29.850117 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:29.764827 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1563) took 1071 ms. Trace:
I20260709 12:24:30.349265 31136 rpcz_store.cc:276] 0709 12:24:28.693049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.693205 (+   156us) service_pool.cc:224] Handling call
0709 12:24:29.764604 (+1071399us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b540006d6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:29.764726 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131716480}
I20260709 12:24:29.699353  1160 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:29.762238 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:29.721491  1123 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:30.149253 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 390000us
I20260709 12:24:29.721356  1074 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:29.799649 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:30.351481 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.355664 31697 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 673) took 1823 ms (client timeout 1551 ms). Trace:
I20260709 12:24:30.352551 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:29.842690  1146 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 12:24:30.356141 31697 rpcz_store.cc:269] 0709 12:24:28.532461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.553288 (+ 20827us) service_pool.cc:224] Handling call
0709 12:24:29.799486 (+1246198us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:30.355594 (+556108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19469696}
W20260709 12:24:30.317891  1172 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
W20260709 12:24:29.765642  1113 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.830s	user 0.000s	sys 0.002s
W20260709 12:24:30.052023  1056 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.002s
I20260709 12:24:30.358698 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:30.011907   864 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:30.347823 31060 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.001s
I20260709 12:24:29.825644  1052 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:29.831606  1165 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:24:30.350632  1168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.002s	sys 0.021s
I20260709 12:24:30.338999   919 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.369721 31060 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:29.841012   600 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.110656  1101 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.371838 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:30.311305  1094 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.373554 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:30.374177 31662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.004s	sys 0.027s
W20260709 12:24:30.384537 31662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.004s	sys 0.027s
I20260709 12:24:30.361701  1155 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:30.387185 30985 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:30.346246   627 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:30.354663  1166 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:30.374068   935 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.010s	sys 0.020s
W20260709 12:24:30.372802 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.009s	sys 0.021s
I20260709 12:24:30.353812  1160 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.386286 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.367609  1173 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:29.765936  1054 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:30.379482  1168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.002s	sys 0.021s
I20260709 12:24:30.389425 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:30.395032 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.355854  1135 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:30.396092 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.009s	sys 0.021s
I20260709 12:24:30.361526  1074 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:30.361665  1123 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:30.436544 31136 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:24:30.436892 31136 pending_rounds.cc:85] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 4
I20260709 12:24:30.437130 31136 pending_rounds.cc:107] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:30.437363 31136 raft_consensus.cc:2889] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:30.437876  1074 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:30.437995  1123 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:30.438428  1074 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.438557  1123 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:30.438922  1074 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.439121  1123 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:30.439744 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.440712 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.417675 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:29.683734  1153 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:30.432806 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 677153us
I20260709 12:24:30.433194 31142 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:30.433434 31141 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:30.386737   975 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:30.433718 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.435101 31139 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:30.435457 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:30.436097 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.448410 31136 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:30.448505 31143 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:30.448662 31140 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:24:30.396493   935 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.010s	sys 0.020s
I20260709 12:24:30.384377  1164 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:30.451208 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:30.451833 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.450412 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:30.455492 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:30.458684 30988 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:30.455518 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:30.459424 30991 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:24:30.448686  1113 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:30.387478   600 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.431555  1129 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:30.461156 30983 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:30.430796  1160 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:30.465695 30987 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:30.467334  1152 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.901s	user 0.016s	sys 0.045s
I20260709 12:24:30.467935  1135 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:30.468474  1135 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:30.470183  1129 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.472462  1129 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:30.392804  1155 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:30.388257   919 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:30.395675  1166 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:30.387321  1101 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:30.476536  1166 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:30.389945   627 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.477299  1123 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:30.478838 31135 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:30.505678 31143 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:30.506628 31137 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:30.489172  1088 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.808s	user 0.008s	sys 0.008s
I20260709 12:24:30.506922 30991 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:30.388131  1094 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.456887   975 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.487331  1155 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:30.519392   627 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:30.498504  1123 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:30.565135  1101 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:30.507383  1088 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:30.569528  1065 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.930s	user 0.013s	sys 0.000s
I20260709 12:24:30.580317 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:30.573449  1074 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:30.589990 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 231338us
W20260709 12:24:30.604621 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.606371 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:30.608003 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:30.609853 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:30.612880 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 259820us
I20260709 12:24:30.582181   627 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:30.613605 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:30.664909 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51395.2us
I20260709 12:24:30.665947 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:30.670780 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 56060.3us
I20260709 12:24:30.671628 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:30.582590  1123 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:30.680641 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:30.681703 31059 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:30.676855 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:30.682801 31063 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:30.680737 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:30.678175 31033 leader_election.cc:400] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9.
I20260709 12:24:30.692763 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:30.600793   864 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:30.705255 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:30.614094  1052 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:30.611502   991 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:30.600286  1166 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:30.610899  1101 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:30.723148 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 272796us
I20260709 12:24:30.611064  1155 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:30.614490  1056 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:30.610918   822 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:30.595348  1164 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:30.702548 31063 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:30.667629  1074 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:30.667320  1173 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:30.662699   919 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:30.701696   953 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:30.671458  1129 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:24:30.710306 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:30.594393  1065 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.843918 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:30.718986 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:30.680987 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.717149  1054 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:30.846735  1189 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.731487  1140 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:30.881448 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.702082   935 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.186s
W20260709 12:24:30.878178 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 195078us
I20260709 12:24:30.879539 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:30.701457   970 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.158s
I20260709 12:24:30.673585   627 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.843427   822 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:30.890290 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 277783us
I20260709 12:24:30.858060  1181 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:30.615036   969 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.230s
I20260709 12:24:30.896210 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:30.723003   991 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.867421  1056 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.732625  1166 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.870499   600 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:30.870857  1052 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:30.737396  1101 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.847203  1155 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:30.986106 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:30.986176 31143 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:30.940744 31065 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:30.987267 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:30.859577  1184 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:30.873418  1185 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:31.009780 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:30.893965 31690 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:30.897070  1179 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.945538   864 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:31.013331  1166 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:31.014209 31064 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:31.017153  1135 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:31.018667   970 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.986769  1181 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.990711   822 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:31.003387 31063 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:30.979537   981 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.939149  1176 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.089457  1179 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:31.092798  1193 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:31.045794   953 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.093248  1173 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:31.019263  1178 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.423s	user 0.000s	sys 0.001s
I20260709 12:24:31.008755   600 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:30.981159  1140 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:31.039999  1094 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:31.119256 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 237705us
W20260709 12:24:31.119516 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 77900.7us
I20260709 12:24:31.120980 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:31.105187 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 392000us
I20260709 12:24:30.949806  1074 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.149s
I20260709 12:24:31.040285  1088 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:31.111711 31662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.011s	sys 0.000s
W20260709 12:24:31.112323 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 125141us
I20260709 12:24:31.113188  1203 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:31.004935   975 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:31.113952  1129 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.099663  1181 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.008487  1052 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:31.107661   769 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.446s	user 0.010s	sys 0.004s
I20260709 12:24:31.158453 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:31.114637  1166 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.130074 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:30.991480  1056 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:31.099833   970 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.115898   953 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.135807  1179 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:31.139137 31662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.011s	sys 0.000s
I20260709 12:24:30.942485  1189 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.101410  1135 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:31.131707 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:31.107510   981 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.108249  1193 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.160864 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:31.183321 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1888) took 1608 ms. Trace:
I20260709 12:24:31.186118 31137 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:31.110749  1176 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:31.122808   600 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.165673 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:31.190526 31143 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:31.144528   935 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:24:31.144855  1094 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:24:31.125319  1140 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.166662 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:31.153576  1054 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:24:31.146399  1088 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:24:31.186196 31144 rpcz_store.cc:276] 0709 12:24:29.574692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.594481 (+ 19789us) service_pool.cc:224] Handling call
0709 12:24:29.627586 (+ 33105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:29.656794 (+ 29208us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:29.656835 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:29.670641 (+ 13806us) log.cc:844] Serialized 26 byte log entry
0709 12:24:30.348480 (+677839us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:30.348537 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.348640 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:31.183127 (+834487us) raft_consensus.cc:1685] finished
0709 12:24:31.183146 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.183266 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9925,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9745,"thread_start_us":645524,"threads_started":1,"wal-append.queue_time_us":826668}
I20260709 12:24:31.189199  1193 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.194694 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:31.195096  1208 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:31.195803 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1852) took 1635 ms. Trace:
I20260709 12:24:31.196426  1209 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.343521 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:31.344440 31693 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:31.343955  1209 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:31.343089 30985 rpcz_store.cc:276] 0709 12:24:29.560448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.562305 (+  1857us) service_pool.cc:224] Handling call
0709 12:24:29.562618 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.388218 (+825600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.195554 (+807336us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b5400226854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:31.195723 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1895842560}
I20260709 12:24:31.346912 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:31.348694 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:31.351344 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 361090us
W20260709 12:24:31.317047  1147 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.832s	user 0.002s	sys 0.000s
W20260709 12:24:31.196534  1197 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:31.197139 31690 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:31.306360 31137 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:31.196575  1203 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:31.185529  1202 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:31.286600 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:31.290971 31140 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:31.195822   981 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:31.165654  1166 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.163383   769 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:31.389433 30985 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:31.166306  1065 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:31.155179  1129 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.316195  1211 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:24:31.317045 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:31.284744  1074 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:31.165465  1198 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.147s
I20260709 12:24:31.306497  1101 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:31.359376 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 200975us
W20260709 12:24:31.367144 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 80666us
I20260709 12:24:31.371568   970 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:31.374687  1202 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:31.376245   953 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:31.378777  1213 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
W20260709 12:24:31.379609  1215 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:31.357311  1183 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:31.393111 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:31.357571   627 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:31.355904   730 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:31.401559 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:31.360726  1208 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:31.443450  1129 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:31.450662 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:31.435334  1195 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:24:31.430302 31694 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:31.434245  1217 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.440716  1140 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:31.441751  1189 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:31.448978  1211 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:31.442348  1052 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:31.371408  1056 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:31.366238   991 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:31.452915 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:31.372156  1185 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.395803  1203 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:31.469466 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:32.032891 30991 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
W20260709 12:24:31.984401  1160 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.484s	user 0.006s	sys 0.008s
I20260709 12:24:32.033826 31064 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:32.033905  1160 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.033957 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:31.713992 31695 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 676) took 3137 ms (client timeout 1529 ms). Trace:
I20260709 12:24:31.516242  1190 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:31.663475  1094 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.036744 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.037279  1094 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:31.578878 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:31.783874 31691 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:31.572599   981 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.034312   730 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:32.037032  1190 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.064131 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:32.039036 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:31.570909 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:31.925719  1181 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.764s	user 0.004s	sys 0.002s
W20260709 12:24:32.035105 31695 rpcz_store.cc:269] 0709 12:24:28.576782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.586388 (+  9606us) service_pool.cc:224] Handling call
0709 12:24:31.713731 (+3127343us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b54005c1a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:31.713899 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2283946880}
I20260709 12:24:32.039790 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:32.078965 31059 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:31.784221 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:32.037988 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:32.067231 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.080355  1190 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:32.065093 31135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 379) took 1608 ms (client timeout 1592 ms). Trace:
I20260709 12:24:32.081013 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:32.060767 30985 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
W20260709 12:24:31.474257 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 328370us
I20260709 12:24:31.418105  1206 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.039300  1183 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:31.614409   769 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:31.432337  1135 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.072479   981 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.063149  1225 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.035988   600 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:31.649314  1164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.008s	sys 0.012s
W20260709 12:24:31.570873  1216 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:24:32.077232 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:31.952857  1117 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:31.921797  1221 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:31.571475  1176 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:31.612922   709 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.035789  1166 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:32.082350 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:32.081084 31135 rpcz_store.cc:269] 0709 12:24:30.433303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.433460 (+   157us) service_pool.cc:224] Handling call
0709 12:24:32.041577 (+1608117us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b5400063954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:32.041699 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60323584}
I20260709 12:24:32.086953  1190 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:32.087462  1054 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:32.089223 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.068194  1140 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.074162 31061 raft_consensus.cc:2410] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:24:32.062897  1030 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.064252  1056 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.072881  1203 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.095140 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.070058   953 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.143213 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:32.106084  1164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.209s	user 0.014s	sys 0.026s
I20260709 12:24:32.088200   952 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.088742  1101 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:32.035274  1211 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.110886 30984 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
W20260709 12:24:32.139544 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 57273.3us
I20260709 12:24:32.153173 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:32.141211 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.025s	sys 0.025s
I20260709 12:24:32.154556 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:32.144019 30985 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:32.145126  1190 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.155738 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.147701  1056 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.156388  1228 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.156809  1228 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.156922 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.157181  1228 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.157694  1056 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.105600   970 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.158209  1088 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:24:32.154644 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.026s	sys 0.025s
W20260709 12:24:32.158998 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 990823us
I20260709 12:24:32.158888   970 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.118170   991 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.161456 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 127595us
I20260709 12:24:31.489040  1074 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.163909 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:32.111023  1209 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.138441  1129 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.141438  1135 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.143641  1176 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.123097  1189 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:32.068652  1217 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.138367   709 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.147150  1164 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:32.082834  1181 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.148540  1054 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:32.212252  1054 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.213021  1181 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:32.160965  1199 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:32.186331 31109 leader_election.cc:336] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.535s (SENT)
I20260709 12:24:32.097376  1225 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.218564  1225 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:32.166875   769 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:32.189438 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.170151   627 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:32.130986  1117 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.193456  1190 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.160142   952 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.160470  1030 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:32.194097  1180 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.675s	user 0.000s	sys 0.011s
I20260709 12:24:32.196806 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 755) took 2679 ms. Trace:
I20260709 12:24:32.198410  1183 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.199079  1094 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:32.255468 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 169750us
I20260709 12:24:32.203826  1211 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.094177  1206 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.206878 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.298525 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:32.139243  1221 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.210592  1074 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.187595  1189 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.214699  1164 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.190245 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:32.238204 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.151201   953 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.236449   769 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:32.179895  1135 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.240015   970 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.240378 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.243188 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:32.168812   991 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.184752   981 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.139508   730 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.241647   709 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:32.294476 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 77889.3us
I20260709 12:24:32.299278 30985 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 2.
I20260709 12:24:32.173744  1129 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:32.331956 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 161423us
I20260709 12:24:32.311183 31068 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
W20260709 12:24:32.249363 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 89582.5us
I20260709 12:24:32.181100  1209 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.114s
W20260709 12:24:32.313580 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 158035us
I20260709 12:24:32.323159 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1691) took 1659 ms. Trace:
I20260709 12:24:32.333133 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:32.335767 31060 rpcz_store.cc:276] 0709 12:24:30.663837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.665045 (+  1208us) service_pool.cc:224] Handling call
0709 12:24:30.665396 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.099458 (+434062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.099487 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:31.099538 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:31.099577 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.322913 (+1223336us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b5400383454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:32.322957 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.323104 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997824896}
I20260709 12:24:32.298839  1160 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:32.338208 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:32.339885 31062 raft_consensus.cc:2410] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:32.340521 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:32.322453   822 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.207s	user 0.012s	sys 0.000s
I20260709 12:24:32.313778  1101 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:32.342120 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.358600 31135 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:32.358770  1101 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.361490 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:32.331606  1193 consensus_meta.cc:287] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.970s	user 0.006s	sys 0.004s
I20260709 12:24:32.352941 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:32.390692 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 50238.7us
I20260709 12:24:32.396162 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.391135 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.395591 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:32.402073 31142 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:32.396914 30988 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:32.402922 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.403541 31141 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:32.404850 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:32.405586 31140 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:32.405812 31664 connection.cc:256] RPC call timeout handler was delayed by -1.9745803979999437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2263s for a response
I20260709 12:24:32.406955 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:32.407743 31695 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:32.408854 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.409745 31698 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:32.410621 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.410823 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.411312 31695 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:32.411533 31690 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 3: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:32.412729 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.414191 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.415861 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:32.416407 31137 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:32.417861 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:32.421118 31140 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:32.422912 31137 raft_consensus.cc:2410] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:24:32.423611 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.424526 31690 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:32.424630 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:32.425292 31695 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:32.425729 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.426039 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.426401 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.426012 31135 raft_consensus.cc:2410] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:24:32.434010 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 81170us
I20260709 12:24:32.434795 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:32.399403  1209 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:32.429740 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 111348us
I20260709 12:24:32.437583 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:32.396248 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:32.394397   627 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:32.436586 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:32.316190  1231 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.121s
I20260709 12:24:32.441555 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.362628  1225 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:24:32.341483  1202 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:32.440696 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:32.255157 31689 rpcz_store.cc:276] 0709 12:24:29.517460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.522185 (+  4725us) service_pool.cc:224] Handling call
0709 12:24:29.574736 (+ 52551us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:29.756437 (+181701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:29.756483 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:31.347152 (+1590669us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b2c000b6068. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:24:31.347985 (+   833us) log.cc:844] Serialized 52 byte log entry
0709 12:24:31.462126 (+114141us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:31.462162 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.462249 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.196510 (+734261us) raft_consensus.cc:1685] finished
0709 12:24:32.196536 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.196747 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86973,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3320,"spinlock_wait_cycles":14580992,"thread_start_us":281954,"threads_started":2,"wal-append.queue_time_us":765727}
I20260709 12:24:32.186869  1208 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.334438  1094 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 12:24:32.429616  1235 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:32.442205 30983 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
W20260709 12:24:32.457062 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 91817us
I20260709 12:24:32.437660  1166 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:32.459450 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:32.461232 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:32.395788   822 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.441555  1190 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:32.439319   627 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.469729 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.470242  1206 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:32.470616  1236 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:24:32.399616  1193 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.451256  1203 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:32.464653  1235 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:32.441992  1241 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:32.480171 31692 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:32.483070  1246 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:32.513019 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49943us
W20260709 12:24:32.488744 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 127341us
I20260709 12:24:32.432024   769 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.456357  1240 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.494720 30988 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:32.475790 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.464969  1166 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.493450  1030 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:32.497826  1189 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.498472  1241 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:32.538296 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.475327   627 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.506541  1235 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:32.532553 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:32.557823  1202 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:32.558311  1202 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:32.558785  1202 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:32.535768 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 139618us
I20260709 12:24:32.495143   970 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:32.562515   970 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.562920   970 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:32.543689 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.540478 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.531524  1246 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.566838  1202 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.567564  1246 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:32.569779 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.501268  1228 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.547051 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:32.537050  1166 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:32.511539 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:32.533243  1240 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.570406 31140 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:32.609611 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.571794 31695 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:32.570787 31135 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:32.610373 31140 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:32.611442 31135 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:32.571883 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:32.572118 31692 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:24:32.572642 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.612599 31141 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:32.573523 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.549084   769 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.578429 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:32.613643 31144 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
W20260709 12:24:32.548434 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 87271.7us
I20260709 12:24:32.586179 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.590623 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:32.618407 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:32.599488 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.576720 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.534948  1054 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
W20260709 12:24:32.612531 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 101084us
I20260709 12:24:32.621120 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:32.540130  1241 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.578364   709 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:24:32.606326 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 146944us
I20260709 12:24:32.595229 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.555567   952 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.554658  1117 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.623881 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:32.625233 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:32.633430 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:32.614936 31141 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:32.576786  1246 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.610358  1240 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.608991   953 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.617175  1160 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:32.619621 31696 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:32.637522 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:32.644343 30988 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:32.632511 31689 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:32.614151  1249 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.616170  1245 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.626708  1247 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:32.655005 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:32.629097  1173 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.626688  1206 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:32.627724  1241 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:32.626248 31138 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:32.632707  1135 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.633239  1217 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.673040 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:32.674085  1250 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.655577  1155 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:32.656904   952 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:32.684231 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:32.687312   822 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.633886   709 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.670800  1243 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.692107 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:32.718968 31142 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:32.685052 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.722393  1217 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:32.722327 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.722941  1217 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:32.723430  1217 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:32.724041 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:32.724982  1230 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:32.726315  1233 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
W20260709 12:24:32.729420 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 91953.4us
I20260709 12:24:32.677106  1160 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:32.731769 31109 leader_election.cc:336] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.692s (SENT)
I20260709 12:24:32.732291 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:32.658232  1231 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 12:24:32.739605  1140 consensus_meta.cc:287] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.000s
I20260709 12:24:32.740253  1183 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.684944  1250 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.684966  1247 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.751626 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:32.760095 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:32.764729 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 152805us
I20260709 12:24:32.765733 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:32.688060  1176 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.669720  1249 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.766342 31061 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:32.769979   709 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.770337  1242 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.145s
I20260709 12:24:32.760422 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:32.756368  1140 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.744652  1228 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:32.758368  1168 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:24:32.682593  1206 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.755259  1250 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.774674  1227 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.746145   991 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.741860  1243 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.732968  1230 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:32.671550  1245 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.900360  1239 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.924422  1239 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.928119 31060 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:32.930297  1186 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.980s	user 0.012s	sys 0.012s
I20260709 12:24:32.934376  1230 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.935447 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 684) took 3886 ms. Trace:
I20260709 12:24:32.935858 31691 rpcz_store.cc:276] 0709 12:24:29.048759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.088182 (+ 39423us) service_pool.cc:224] Handling call
0709 12:24:31.714198 (+2626016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.072161 (+357963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.935250 (+863089us) spinlock_profiling.cc:243] Waited 2.6 s on lock 0x7b54005c1a50. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:32.935384 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5977650944}
I20260709 12:24:32.936615  1230 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.941133  1250 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:32.943899 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:32.944594  1245 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:32.944710 30990 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.945179  1245 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.945657  1245 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.946326  1199 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:32.862538 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60438.6us
I20260709 12:24:32.947414 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1988) took 1466 ms. Trace:
I20260709 12:24:32.819860  1135 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:24:32.947878 30989 rpcz_store.cc:276] 0709 12:24:31.481141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.481329 (+   188us) service_pool.cc:224] Handling call
0709 12:24:31.481703 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.078234 (+596531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.078280 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:32.078336 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:32.078381 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.947116 (+868735us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b5400227254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:32.947191 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.947342 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371318144}
I20260709 12:24:32.948695  1135 consensus_queue.cc:1243] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:32.949206 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.934065 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 383) took 2499 ms. Trace:
I20260709 12:24:32.772536  1181 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.958555 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.959193 30989 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:32.961514 31064 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:32.901765 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:32.748219  1030 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.756156  1074 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:32.775696  1193 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:32.937801  1217 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:32.943922  1185 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.875s	user 0.009s	sys 0.001s
I20260709 12:24:32.949223  1135 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:32.747958   953 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:32.910346 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 332141us
I20260709 12:24:32.977533 30958 leader_election.cc:400] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6.
I20260709 12:24:32.977888 30958 leader_election.cc:403] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:32.910043  1243 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:32.912659   822 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.980391 30990 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:32.981397 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:32.984223 31143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:32.914458  1246 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:32.919335  1153 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:32.907267  1227 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.952016 31139 rpcz_store.cc:276] 0709 12:24:30.434338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.456140 (+ 21802us) service_pool.cc:224] Handling call
0709 12:24:30.456423 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.456738 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.456765 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:30.461772 (+  5007us) log.cc:844] Serialized 26 byte log entry
0709 12:24:30.577126 (+115354us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:30.851155 (+274029us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.092194 (+241039us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b180025c240. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:24:31.127208 (+ 35014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.127326 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.929443 (+1802117us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:32.933844 (+  4401us) raft_consensus.cc:1685] finished
0709 12:24:32.933862 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.933982 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18267,"prepare.queue_time_us":24051,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":166527,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":6131,"spinlock_wait_cycles":79688320,"thread_start_us":634614,"threads_started":3,"wal-append.queue_time_us":400232}
I20260709 12:24:32.964913 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:32.965996 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 64502.6us
I20260709 12:24:32.905324  1228 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:32.832934 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 72604.6us
I20260709 12:24:32.863685 31141 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 5. Ops: []
I20260709 12:24:32.973726 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 690) took 2375 ms. Trace:
I20260709 12:24:32.968473  1074 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:32.961619  1181 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.990254 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:33.522415 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:33.522657 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:33.519961  1235 consensus_meta.cc:287] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.911s	user 0.009s	sys 0.000s
W20260709 12:24:33.502617  1166 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.889s	user 0.009s	sys 0.000s
W20260709 12:24:33.523708 30991 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1993) took 1667 ms (client timeout 1874 ms). Trace:
I20260709 12:24:33.523991  1166 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:33.524225 30991 rpcz_store.cc:269] 0709 12:24:31.856552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.033933 (+177381us) service_pool.cc:224] Handling call
0709 12:24:33.522490 (+1488557us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:33.523574 (+  1084us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41315072}
I20260709 12:24:33.524902  1235 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:33.525556 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.526067  1166 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:33.022670   822 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:33.466727  1243 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:33.314992  1239 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.985882  1241 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.981884  1250 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:32.980935  1129 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:33.186120 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.971516  1263 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:32.996255 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1986) took 1584 ms. Trace:
I20260709 12:24:33.529266 31061 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:32.986240  1187 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:32.971626  1140 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:33.531476 31143 raft_consensus.cc:606] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:24:33.593674 30986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.021s	sys 0.044s
I20260709 12:24:32.972797  1245 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:33.594081 30986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.021s	sys 0.044s
I20260709 12:24:33.595355 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 1990) took 2034 ms. Trace:
I20260709 12:24:32.974932  1185 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.973811 31138 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:33.466956   769 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.847s	user 0.011s	sys 0.000s
I20260709 12:24:33.538528  1181 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.595700 30986 rpcz_store.cc:276] 0709 12:24:31.560689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.563911 (+  3222us) service_pool.cc:224] Handling call
0709 12:24:32.987045 (+1423134us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:32.987255 (+   210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.987482 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:32.987776 (+   294us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.595011 (+607235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:33.595061 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:33.595111 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.595173 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.595275 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":826,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"spinlock_wait_cycles":7378688,"thread_start_us":606927,"threads_started":1}
I20260709 12:24:32.971573  1249 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:33.529515 31697 rpcz_store.cc:276] 0709 12:24:30.598732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.610841 (+ 12109us) service_pool.cc:224] Handling call
0709 12:24:30.671015 (+ 60174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.671243 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:30.871638 (+200395us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.973216 (+2101578us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b18003dd260. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a
0709 12:24:32.973385 (+   169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:32.973433 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:32.973486 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.973575 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.973680 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":466359,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":2343,"raft.queue_time_us":1357,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":13165,"spinlock_wait_cycles":5491456,"thread_start_us":438628,"threads_started":2,"wal-append.queue_time_us":282255,"wal-append.run_cpu_time_us":33739,"wal-append.run_wall_time_us":1877791}
I20260709 12:24:33.581908  1189 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:33.022042  1190 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.001s
I20260709 12:24:33.500226  1217 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:32.971432  1030 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:33.188793   970 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.000s
I20260709 12:24:33.591162 30990 rpcz_store.cc:276] 0709 12:24:31.411887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.488706 (+ 76819us) service_pool.cc:224] Handling call
0709 12:24:32.943714 (+1455008us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:32.996172 (+ 52458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7960320}
I20260709 12:24:33.526074  1228 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.593194 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.121917   709 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:33.534804 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 553491us
I20260709 12:24:33.123912  1061 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:33.583226 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:33.538657  1227 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.597937 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.667706 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:33.668232 30986 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:33.598536 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.671133 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:32.981771  1232 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6.
I20260709 12:24:32.982390  1206 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:33.324699  1154 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:33.673198  1154 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:32.975889  1135 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:33.594053 31143 raft_consensus.cc:614] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:33.533282  1153 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:33.588754 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.679781 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 411) took 2487 ms. Trace:
I20260709 12:24:33.679841 30989 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:33.680133 31694 rpcz_store.cc:276] 0709 12:24:31.193088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.193274 (+   186us) service_pool.cc:224] Handling call
0709 12:24:31.285198 (+ 91924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.040800 (+755602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.679539 (+1638739us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54006ee6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:33.679714 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328732544}
I20260709 12:24:33.592839  1246 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:33.593899  1140 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:33.603469  1190 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:33.606612 31693 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:24:33.019984  1240 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:33.607070  1235 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:33.611954 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:33.686019 30983 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.686844  1240 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:33.689744 30986 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:33.602433  1189 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:33.691645  1189 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:33.596948  1187 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:33.646867   709 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:33.693930   709 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:33.649501   822 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:33.651502  1030 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:33.594422  1263 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:33.614840 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:32.984186   627 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:33.613863 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 388) took 1273 ms. Trace:
I20260709 12:24:33.638834  1228 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:33.651530  1241 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:33.657155 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:33.593010  1239 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:24:33.662395 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:33.663422 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 80261.2us
I20260709 12:24:33.614092   970 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:33.663977  1208 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:33.639145  1074 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:33.666455 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:33.740960 31139 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:33.680737 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.741801 30989 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:33.742586 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.743057 30989 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 3: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:33.699348 31137 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:33.712779 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:33.713429 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:33.713897 30983 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:33.706833 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.716408 30991 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:33.718518 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:33.679236  1209 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.215s	user 0.011s	sys 0.014s
I20260709 12:24:33.723654 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:33.721691 31136 rpcz_store.cc:276] 0709 12:24:32.340849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.340993 (+   144us) service_pool.cc:224] Handling call
0709 12:24:33.613148 (+1272155us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fe9c 00007f2ee876fa5b 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:33.613307 (+   159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.613471 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.613497 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:33.613535 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:33.613571 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.613650 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.613791 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2534016}
I20260709 12:24:33.748755 31058 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:33.749224 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:33.750028 31136 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:33.750284 31063 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:33.684242  1140 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.738176 31141 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:33.753357 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.684041  1246 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:33.754084 30989 raft_consensus.cc:2393] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:33.745575 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 218704us
I20260709 12:24:33.754876 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.760229 30983 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:33.760993 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:33.761226 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:33.761518 30983 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.757444 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:33.765875 30983 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:33.766928 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.768426 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 438) took 1363 ms. Trace:
I20260709 12:24:33.768739 30989 rpcz_store.cc:276] 0709 12:24:32.404763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.767746 (+1362983us) service_pool.cc:224] Handling call
0709 12:24:33.767996 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.768148 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.768171 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:33.768211 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:33.768241 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.768289 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.768380 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:33.755162 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.755630 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:33.757305 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:33.771775 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:33.772055 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:33.772446 31141 raft_consensus.cc:2368] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3.
I20260709 12:24:33.772637 31136 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
W20260709 12:24:33.748353  1209 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.011s	sys 0.014s
I20260709 12:24:33.773340 31058 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:33.774305 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:33.773939 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:33.775029 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 238877us
I20260709 12:24:33.743685  1153 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:33.775918 30983 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:34.139535 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.427225 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 436) took 2026 ms. Trace:
I20260709 12:24:34.427644 30983 rpcz_store.cc:276] 0709 12:24:32.400948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.775470 (+1374522us) service_pool.cc:224] Handling call
0709 12:24:33.775715 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.426955 (+651240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.427166 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.428421  1153 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:34.428635  1140 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.000s
I20260709 12:24:33.788880 31136 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:34.429399  1140 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:33.776479  1209 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:34.432896  1209 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:34.046972  1181 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.436048  1153 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:34.436456  1181 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:34.436641  1153 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:33.758736  1074 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:33.851370 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 77156.9us
I20260709 12:24:33.794329  1243 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:33.825554 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:33.797912 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.837909 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 80531.3us
I20260709 12:24:34.445684  1074 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:34.446349 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.447904 31058 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:34.449067 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:34.449352 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.449645 31058 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.787865  1154 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:34.450853 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 375877us
I20260709 12:24:34.456605  1153 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:34.457149  1293 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:34.458662 31063 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:34.461614 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.476580 31059 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:34.176658 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.477764 31135 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
W20260709 12:24:33.813097 31109 leader_election.cc:336] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.962s (SENT)
I20260709 12:24:34.478678 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:34.479130 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:33.988344   627 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:34.123123  1208 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:34.495389 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 682286us
I20260709 12:24:33.869132  1278 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:33.955411  1277 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:33.989552  1263 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:34.496555 31109 leader_election.cc:336] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.026s (SENT)
I20260709 12:24:34.473292  1243 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:34.501188 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:34.496806 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:34.474045  1241 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:34.463887  1140 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:34.499158 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.508028 31058 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:34.508438 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:34.509217 31140 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:34.516269  1268 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:34.516810  1268 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:34.517969   970 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:34.518603   970 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:34.520817 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:34.498322  1277 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.499667  1030 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:34.497344  1290 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:34.497886  1278 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:34.485008  1293 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:34.514124  1222 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 2.892s	user 0.011s	sys 0.004s
I20260709 12:24:34.511785  1263 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:34.509915   627 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:24:34.507913  1243 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.535244  1243 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:34.535739  1293 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:34.536091 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 768) took 2044 ms. Trace:
I20260709 12:24:34.536505 31689 rpcz_store.cc:276] 0709 12:24:32.491680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.493643 (+  1963us) service_pool.cc:224] Handling call
0709 12:24:34.535902 (+2042259us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b54004a24d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:34.536034 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103607168}
I20260709 12:24:34.525588  1290 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:34.539182  1290 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.545799 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2004) took 2293 ms. Trace:
I20260709 12:24:34.546305 31064 rpcz_store.cc:276] 0709 12:24:32.252190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.252303 (+   113us) service_pool.cc:224] Handling call
0709 12:24:34.545503 (+2293200us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54000c6f54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:34.545700 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522375424}
I20260709 12:24:34.550082  1208 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.555600 31140 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
W20260709 12:24:34.536904  1228 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.797s	user 0.009s	sys 0.001s
I20260709 12:24:34.567525 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.517810  1268 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:34.573343 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 341231us
W20260709 12:24:34.573611 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 68356.6us
I20260709 12:24:34.525208  1277 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:34.574810 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 67774.1us
I20260709 12:24:34.511147  1241 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:34.589982 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:34.578325  1290 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:34.592468 30992 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1991) took 3006 ms (client timeout 2032 ms). Trace:
W20260709 12:24:34.603474 30992 rpcz_store.cc:269] 0709 12:24:31.564202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.564316 (+   114us) service_pool.cc:224] Handling call
0709 12:24:34.567343 (+3003027us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:34.570249 (+  2906us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7132544}
W20260709 12:24:34.532855  1227 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.812s	user 0.012s	sys 0.000s
I20260709 12:24:34.513073  1074 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:24:34.528226  1278 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:34.605259 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.594456 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:34.604755  1227 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.530882  1263 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:34.607069 31140 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:34.606228 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:34.612530 31064 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.609601 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:34.604380  1299 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:34.615290 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:34.617082 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:34.593590   970 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 12:24:34.621966 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.622459 30955 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.66:45157 timed out after 1.615s (SENT) [suppressed 12 similar messages]
I20260709 12:24:34.624420 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.625000 31063 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:34.625102 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.625869 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.626256 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.652480 31063 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:34.652561 31059 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:34.626554 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.626891 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.653342 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:34.613907 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 117406us
I20260709 12:24:34.653999 31063 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:34.630330 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:34.616058  1088 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:34.634764   901 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:34.655685 31068 raft_consensus.cc:2410] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:34.635437 31139 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:34.607712  1268 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.582333  1228 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:34.673050 31033 connection.cc:256] RPC call timeout handler was delayed by -1.6943023660001018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
I20260709 12:24:34.673439 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.636412  1227 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:34.637048  1241 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:34.643872 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.644853 31689 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:34.645541 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.646497 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.647209 31695 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:34.648641 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.649269 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.649956 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.653039 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:34.627393  1202 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:34.671497  1299 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:34.692549 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:34.654559 30987 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:34.659336 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:34.662829 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.663044 31068 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:24:34.674770  1221 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:34.665899 31064 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:34.701396 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:34.666146 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.666011 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.702813 31136 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:34.703613 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:34.704115 31136 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:34.703929 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:34.613876  1263 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:34.673439 31033 connection.cc:256] RPC call timeout handler was delayed by -1.6871023660000901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
I20260709 12:24:34.684862 31140 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:34.677413  1265 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:34.627594 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.689291 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.689908 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:34.708379 31696 raft_consensus.cc:2368] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6.
I20260709 12:24:34.690794 31694 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:34.672571   864 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:34.665050  1154 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:34.698520 30992 raft_consensus.cc:2368] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:34.715487 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.716187 31064 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:34.717237 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:34.697697 31664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.054s	sys 0.033s
I20260709 12:24:34.699048 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:34.737740 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:34.737920 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.700100 31698 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:34.739305 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:34.705489 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:34.740025 30991 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:34.691653   901 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:34.708967 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.706073  1030 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:34.710520 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.741421 31068 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:34.675588  1295 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:34.708557  1202 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:24:34.784082 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.770004 31136 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
W20260709 12:24:34.816452 31137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1893) took 2247 ms (client timeout 1950 ms). Trace:
I20260709 12:24:34.806178 30987 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:34.725142 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:34.842655 31664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.059s	sys 0.039s
I20260709 12:24:34.746555 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:34.815837 30988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 432) took 2476 ms (client timeout 1538 ms). Trace:
I20260709 12:24:34.718695 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.847137 30986 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 5. Ops: [3.3-3.3]
W20260709 12:24:34.814147 31060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2005) took 2054 ms (client timeout 1978 ms). Trace:
I20260709 12:24:34.849803 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:34.849639 31060 rpcz_store.cc:269] 0709 12:24:32.759637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.759782 (+   145us) service_pool.cc:224] Handling call
0709 12:24:34.813965 (+2054183us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54002cf1d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:34.814081 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384573184}
I20260709 12:24:34.742897 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:34.851871 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.852283 31140 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:34.852380 31060 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:34.852845 31065 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:34.853242 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.853677 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.854239 31065 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:34.855084 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:34.845908 30988 rpcz_store.cc:269] 0709 12:24:32.339244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.403367 (+ 64123us) service_pool.cc:224] Handling call
0709 12:24:34.815688 (+2412321us) spinlock_profiling.cc:243] Waited 56.4 ms on lock 0x7b540011fdd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:34.815778 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129754880}
W20260709 12:24:34.844714 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 1987) took 3380 ms (client timeout 2191 ms). Trace:
I20260709 12:24:34.855980 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.856554 30988 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:34.856709 31065 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:34.857973 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.711812  1154 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:34.858026 31698 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:34.894040 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.742887  1299 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:34.815145 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 1992) took 3315 ms. Trace:
I20260709 12:24:34.895931 31067 rpcz_store.cc:276] 0709 12:24:31.499775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.499933 (+   158us) service_pool.cc:224] Handling call
0709 12:24:31.571510 (+ 71577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.173943 (+602433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.173976 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:32.174015 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:32.174052 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.814917 (+2640865us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54000fc654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:34.814958 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.815070 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384984320}
I20260709 12:24:34.896538 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.897735   864 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.897836  1296 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.898335   864 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.898525  1296 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.899290 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:34.900803   864 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:24:34.901571   953 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:34.900849  1296 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:34.856140 30987 rpcz_store.cc:269] 0709 12:24:31.463911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.464517 (+   606us) service_pool.cc:224] Handling call
0709 12:24:34.630175 (+3165658us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:34.844637 (+214462us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8899840}
I20260709 12:24:34.757460 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.903862 30991 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:34.904872 31695 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:34.882906 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.906153 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:34.844105 31137 rpcz_store.cc:269] 0709 12:24:32.569065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.571678 (+  2613us) service_pool.cc:224] Handling call
0709 12:24:34.816278 (+2244600us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000be5d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:34.816364 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468018560}
I20260709 12:24:34.907287 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.906004 31061 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:34.907610 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:34.908842 31137 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:34.908106 31060 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:34.906034 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:34.920516 31068 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:34.910384 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.709628  1241 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:34.911842 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.859051 30984 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 7. Ops: [5.4-5.4]
I20260709 12:24:34.890513 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.923692 31136 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:34.922531 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.924440 30991 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:34.924641 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.925213 31136 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:34.925257 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:34.926255 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.918533 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:34.946283 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.947106 30983 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:34.844275  1030 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:34.926213 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:34.928284 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.931932 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:34.956770 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.957398 31696 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:34.957607 31136 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:34.790549   901 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:34.957701 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:34.958259 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1598) took 1186 ms. Trace:
I20260709 12:24:34.958592 31136 rpcz_store.cc:276] 0709 12:24:33.771392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.957067 (+1185675us) service_pool.cc:224] Handling call
0709 12:24:34.957329 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.958019 (+   690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.958184 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.932752 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.960888 31140 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:34.756028 31690 raft_consensus.cc:2410] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:34.940129 31661 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:34.940778 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:34.962071 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:34.962455  1208 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:34.962918  1208 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.963286  1208 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:34.963414   970 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:34.963608  1239 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:34.963930   970 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.964090  1239 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:34.964399   970 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:34.965029  1239 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.941329 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:34.966895 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:34.942045 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 235670us
I20260709 12:24:34.962009 31068 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:34.967572 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.916975 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.961159 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:34.969202 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.970700 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:34.948119 30991 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:34.945004 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.742624 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:34.808630  1228 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:34.955054 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:34.955571 31065 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:34.937362 31137 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:34.974584 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:34.975303 31695 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:34.975477 31065 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:34.971691 31698 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 3.
I20260709 12:24:34.949595 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:34.955523  1234 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.493s	user 0.005s	sys 0.016s
I20260709 12:24:34.715081  1265 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:34.975926 31033 connection.cc:256] RPC call timeout handler was delayed by -1.6016023660004066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 12:24:34.976995 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:34.986475 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:34.955277  1241 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:34.980091 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:34.980649 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.981262   981 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:34.955025  1290 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:34.978664  1274 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:34.799752  1278 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:34.988161 31136 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:34.988857  1164 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:34.994438 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1601) took 1126 ms. Trace:
W20260709 12:24:34.992764 31033 connection.cc:256] RPC call timeout handler was delayed by -1.432402366000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
I20260709 12:24:34.995178 31136 rpcz_store.cc:276] 0709 12:24:33.867801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.975238 (+1107437us) service_pool.cc:224] Handling call
0709 12:24:34.987935 (+ 12697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.994192 (+  6257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.994387 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.973511  1239 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.990302 31698 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:34.992594 31691 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:34.961102  1030 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:34.972677   953 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:34.983206  1234 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.539s	user 0.005s	sys 0.016s
I20260709 12:24:34.821877  1135 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.982033 31144 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:35.000504   901 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.003448 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1599) took 1225 ms. Trace:
I20260709 12:24:35.003824 31144 rpcz_store.cc:276] 0709 12:24:33.777837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.967887 (+1190050us) service_pool.cc:224] Handling call
0709 12:24:34.979779 (+ 11892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.003203 (+ 23424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.003393 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.825099  1123 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.007755 31060 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:34.992815 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:34.950402  1263 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:34.995076  1274 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:34.995559 31033 connection.cc:256] RPC call timeout handler was delayed by -1.328002366000419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23570000000000002s for a response
I20260709 12:24:34.997117  1305 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:24:35.017136 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:35.017630  1241 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:34.991832  1154 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.019286  1209 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.019714  1209 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:35.020100  1209 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.021515 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:34.994585 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:35.031105 31692 raft_consensus.cc:2368] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:35.031307 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.033020  1263 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.034363  1290 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.034806  1290 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:35.035202  1290 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:35.038784 31062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 1995) took 3398 ms (client timeout 2030 ms). Trace:
W20260709 12:24:35.039310 31062 rpcz_store.cc:269] 0709 12:24:31.640510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.640751 (+   241us) service_pool.cc:224] Handling call
0709 12:24:35.038526 (+3397775us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54003e6054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:35.038698 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564755328}
I20260709 12:24:35.042644 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1902) took 2268 ms. Trace:
I20260709 12:24:34.992488   627 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.998528 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:34.990864  1295 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:35.002003 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:35.044750 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.045184 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.047358 31059 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:35.046254  1263 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:35.051280 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 52828.9us
I20260709 12:24:35.052706 30958 leader_election.cc:400] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6.
I20260709 12:24:35.053445 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:35.054260 31692 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:35.054771  1278 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.053109 30958 leader_election.cc:403] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:24:35.056349 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:35.057512  1155 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6.
I20260709 12:24:35.058192  1155 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:35.059494 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:35.063097  1277 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.063519  1221 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:35.063535  1155 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:35.064055  1221 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:35.065008 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:35.065232  1277 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:35.064896  1221 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.014680  1274 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.070739  1221 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:35.071476 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.052145   627 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.076494   627 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.076957   627 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:35.077378   627 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.078632 31141 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:35.080386 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1586) took 1419 ms. Trace:
I20260709 12:24:35.080752 31141 rpcz_store.cc:276] 0709 12:24:33.660640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.787993 (+127353us) service_pool.cc:224] Handling call
0709 12:24:34.425679 (+637686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.078998 (+653319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.080160 (+  1162us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b540006d6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:35.080328 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1501202688}
I20260709 12:24:35.081378 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:35.052067 31690 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:35.084770  1155 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:35.081871 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:35.086683 30958 leader_election.cc:400] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3.
I20260709 12:24:35.065743  1295 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:35.066287   730 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.066746 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.087253 31141 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:35.087013 30958 leader_election.cc:403] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:35.099836 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:35.008481  1243 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:24:35.104367 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 117151us
I20260709 12:24:35.110464 31109 leader_election.cc:400] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:35.110802 31109 leader_election.cc:403] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:24:35.112457 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:35.113075 31141 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:35.046255 31138 rpcz_store.cc:276] 0709 12:24:32.773920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.774089 (+   169us) service_pool.cc:224] Handling call
0709 12:24:32.864234 (+ 90145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.603070 (+738836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.042350 (+1439280us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b540006e0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:35.042570 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118785408}
I20260709 12:24:35.052605  1268 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:35.100644  1198 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:35.116688 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:35.080390  1221 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:35.115658 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:35.129141 31692 raft_consensus.cc:2368] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:35.130128  1202 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:35.128181 31068 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:35.132229 31065 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:35.135427 31141 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:35.111267  1305 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 2. Current term is 3.
I20260709 12:24:35.107128  1228 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:35.116317  1277 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:35.144479 31068 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:35.146744 31068 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:35.152364 31138 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:35.152228   901 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:35.154704  1295 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.116367  1227 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:35.126405  1198 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:35.159433 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45906.2us
I20260709 12:24:35.163400 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:35.108723  1164 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:35.137528  1239 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:35.133315  1221 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.140475  1268 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:35.179425  1135 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.220352 31140 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:35.183127 31697 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 12:24:35.221007 31697 pending_rounds.cc:85] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 7
I20260709 12:24:35.221282 31697 pending_rounds.cc:107] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 12:24:35.221671 31697 raft_consensus.cc:2889] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:35.191039 31065 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:35.197981 31059 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:35.225220 31059 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:35.176362  1168 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:24:35.227651 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 55975.4us
W20260709 12:24:35.202487 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.208427 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.228343 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.228579 31065 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:35.228915 30989 raft_consensus.cc:2376] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7. Re-sending same reply.
I20260709 12:24:35.212080 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:35.215302 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.141650   981 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:35.229751 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:35.218411 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:35.204214 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:35.182407 31693 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 772) took 2677 ms (client timeout 2148 ms). Trace:
I20260709 12:24:35.209643 30983 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:35.231370 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:35.231765 30983 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:35.232358 31059 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:35.230006 30992 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:35.217053 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 53761.6us
I20260709 12:24:35.164801  1198 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:35.234867 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:35.237491 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:35.248394 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:35.249155 31142 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:35.218940  1268 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:35.251950  1268 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.254858  1265 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.255385  1265 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:35.200657  1239 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:35.218672  1221 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.198992  1187 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.259469 31139 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:35.164676  1176 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
W20260709 12:24:35.290143 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48451.8us
W20260709 12:24:35.287151 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 47655.3us
I20260709 12:24:35.299625 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.300673 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.157675  1228 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.289666 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:35.166682  1227 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.256356  1278 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:35.237079 31693 rpcz_store.cc:269] 0709 12:24:32.504806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.531627 (+ 26821us) service_pool.cc:224] Handling call
0709 12:24:35.182151 (+2650524us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54005e7754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:35.182328 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223951104}
I20260709 12:24:35.257422  1140 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:35.234512  1310 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:35.321918  1310 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.285789  1239 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:35.325277 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:35.326184 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 220734us
I20260709 12:24:35.327231 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:35.330853 31142 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:35.333250  1268 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:35.334124 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90771.1us
W20260709 12:24:35.335278 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 212315us
I20260709 12:24:35.336647 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.343806 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:35.306874   952 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:35.322412  1310 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:35.352020 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 51405.1us
I20260709 12:24:35.342541 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:35.348723 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.305080  1187 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.354259 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.358385 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:35.303280 31059 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:35.361634 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:35.370385  1265 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:35.370816  1265 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:35.373996  1296 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:35.374471  1203 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:35.374872  1203 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:35.375237  1203 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:35.342124   970 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.377776   970 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.288527  1208 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.326866   627 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.381649 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.335073  1176 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:35.381738 31141 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:35.335325 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.386197 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:35.387430 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.389089  1277 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:35.389540  1277 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.389905  1277 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:35.398860   709 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:35.399377   709 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:35.399839   709 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.421278 31142 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:35.324607  1281 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:24:35.410493 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.361593  1310 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:35.448237 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 60868.4us
I20260709 12:24:35.362656   981 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.457782 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:35.457984  1313 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 12:24:35.440713  1140 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:35.461323   600 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.460573  1140 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.409904 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.470587 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.471269 31689 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
W20260709 12:24:35.472368  1319 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:24:35.461859   600 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.384809  1176 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:35.379863  1187 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.474722 30990 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:24:35.475428 31698 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:35.323241  1164 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.476210  1187 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:35.397437 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.409561  1241 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:35.474134 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 398) took 1019 ms. Trace:
I20260709 12:24:35.479621  1327 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:35.480235  1327 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:35.479283 31144 rpcz_store.cc:276] 0709 12:24:34.454595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.004598 (+550003us) service_pool.cc:224] Handling call
0709 12:24:35.004917 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:35.103892 (+ 98975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.103935 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:35.115697 (+ 11762us) log.cc:844] Serialized 26 byte log entry
0709 12:24:35.190505 (+ 74808us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:35.190561 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.190649 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.473866 (+283217us) raft_consensus.cc:1685] finished
0709 12:24:35.473891 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.474067 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11798,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":76,"thread_start_us":120930,"threads_started":2,"wal-append.queue_time_us":2381}
I20260709 12:24:35.372066  1265 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.435249   709 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.396018  1269 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:35.506178 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 148353us
I20260709 12:24:35.474154  1176 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.510567  1265 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.514343 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:35.524435 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:35.540974 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 225056us
W20260709 12:24:35.551343 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 189264us
W20260709 12:24:35.554941  1030 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.006s
I20260709 12:24:35.457322 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:35.525266 31068 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:35.553150 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:51358 (request call id 27) took 2567 ms. Trace:
W20260709 12:24:35.553826 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.555438  1030 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.558229 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:35.558988 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 101742us
I20260709 12:24:35.561728 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:35.563289 31109 leader_election.cc:400] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:35.555790 31143 rpcz_store.cc:276] 0709 12:24:32.982536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.984089 (+  1553us) service_pool.cc:224] Handling call
0709 12:24:35.549257 (+2565168us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b54001e9bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88e853 00007f2ef085c60a 00007f2ee9de27d4 00007f2ee9de2762 00007f2ee9de2691 00007f2ee9de260c 00007f2ee9de18c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:35.549736 (+   479us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9505536}
I20260709 12:24:35.526235  1263 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.549252  1278 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:35.572157 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:35.564075 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:35.526468  1187 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:35.575867 31135 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:35.581115 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:35.566609  1176 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.581914 31065 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:35.567929   970 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.550594  1203 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.504866  1327 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:35.605551  1328 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:35.578374  1332 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:35.610477  1332 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.610325  1278 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.610894  1332 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:35.579859  1310 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.615649  1203 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.616021  1310 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.596362  1339 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:24:35.622298  1227 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.564917  1337 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:24:35.623809  1209 consensus_meta.cc:287] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.000s
I20260709 12:24:35.624212  1209 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.626711 31061 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:35.624552  1274 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.554s	user 0.004s	sys 0.004s
W20260709 12:24:35.629977 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 72766.8us
I20260709 12:24:35.571082  1331 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:35.584762   970 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.584427  1187 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:35.599316  1299 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:35.654486 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:35.584787  1335 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:35.661231 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.655472 31144 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:24:35.634716 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.636636 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:35.656078 30992 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:35.632330  1198 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.636777 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:35.657943 30984 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
W20260709 12:24:35.657986 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 165629us
I20260709 12:24:35.632629  1329 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:35.660641 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.664093 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1720) took 1038 ms. Trace:
I20260709 12:24:35.680189 30987 raft_consensus.cc:2376] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3. Re-sending same reply.
I20260709 12:24:35.680868 31060 rpcz_store.cc:276] 0709 12:24:34.626085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.928169 (+302084us) service_pool.cc:224] Handling call
0709 12:24:34.928468 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.008273 (+ 79805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.663813 (+655540us) spinlock_profiling.cc:243] Waited 79 ms on lock 0x7b54002a4754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:35.664021 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181774208}
W20260709 12:24:35.665433 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.670971 30986 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:35.657902   970 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.656795   864 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:24:35.729915  1346 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:35.730580  1187 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.732118  1030 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:35.733193 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:35.734922 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.659734  1332 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.659938  1228 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.633627  1221 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.724326  1239 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.632195  1274 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.801833 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:35.804776 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1717) took 1179 ms. Trace:
I20260709 12:24:35.740319 30988 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:35.841286 31143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:35.844945 31143 raft_consensus.cc:606] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:35.845242 31136 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:35.845381 31143 raft_consensus.cc:614] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:35.844942 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 480039us
I20260709 12:24:35.845348 31696 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:35.738902 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:35.839685 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 275652us
I20260709 12:24:35.842550 31058 rpcz_store.cc:276] 0709 12:24:34.625366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.654091 (+ 28725us) service_pool.cc:224] Handling call
0709 12:24:34.848578 (+194487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.848757 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.848782 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.848823 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:34.848861 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.848913 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.804554 (+955641us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54002cf1d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:35.804705 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395947392}
I20260709 12:24:35.847481 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.848755 31109 leader_election.cc:400] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:35.849376 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.736601 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.844556 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:35.846277 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:35.854391 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 270657us
W20260709 12:24:35.773015  1290 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.737s	user 0.007s	sys 0.003s
I20260709 12:24:35.856420 31059 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:35.735599 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.851791  1074 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.857319  1346 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:35.857802  1346 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:35.857652  1074 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.858142 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:35.858238  1346 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.859781  1074 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.860201  1239 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.860714  1030 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:35.861191  1296 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.861589  1296 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.861915  1296 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.862402  1221 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.863040 30955 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:35.863919  1221 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:35.864131 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:35.864295  1221 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:35.864737  1221 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:35.865363 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:35.865491  1277 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.853356  1242 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:24:35.867177 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:35.867538  1242 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:35.867914  1242 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:35.867970 30955 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:35.860253  1209 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.868368  1332 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:35.772037  1337 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:35.869464  1311 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.840571   709 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.866525 30991 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:35.867198 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.871371 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:35.871641 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:35.868346  1242 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:35.800103  1246 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.797266   901 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:35.890208   901 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:35.891147 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.891062   901 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:35.872232 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.892266 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.893285 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.739979  1198 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.894549 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.895745 30987 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:35.896031 30987 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:35.874074 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:35.850247  1140 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.897384  1140 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.850445  1327 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.882761  1337 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.898715  1327 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.899977  1140 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.900324  1337 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:35.902560 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:35.888286 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:35.878082 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.850602  1299 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:35.884567 31143 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:35.853470  1312 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:24:35.886116  1346 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.886773 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.884258  1203 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:35.906343  1202 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:24:35.912978  1203 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:35.888340  1135 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.002s
I20260709 12:24:35.888734 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:35.868861  1243 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:35.906531 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:35.907090 31144 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:35.884482  1056 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.876956  1155 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.909672 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.904877 30987 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.884807  1268 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.900694  1327 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.880082  1311 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:35.911298 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:35.911394  1281 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.911819   730 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:35.912158  1346 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.912343  1299 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:35.918238  1168 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:35.873764  1290 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.908515 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:35.922974  1198 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.935210  1198 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.924561 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:35.940017 31143 raft_consensus.cc:2376] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8. Re-sending same reply.
I20260709 12:24:35.924602 31136 raft_consensus.cc:606] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.941022 31136 raft_consensus.cc:614] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:35.926942 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 704) took 1269 ms. Trace:
I20260709 12:24:35.942054 31697 rpcz_store.cc:276] 0709 12:24:34.657350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.688774 (+ 31424us) service_pool.cc:224] Handling call
0709 12:24:35.182794 (+494020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.222200 (+ 39406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.926742 (+704542us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54005e7750. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:35.926883 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135530368}
I20260709 12:24:35.923894  1155 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:35.926493 31137 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:35.944553 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.929520 31062 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:35.945212 31696 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
W20260709 12:24:35.938733 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.925485 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:35.946098 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.932837 30987 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:35.949800 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.950790 31696 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:35.950695 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:35.951462 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:35.947254  1155 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.953824 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:35.921221 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:35.955650 30983 raft_consensus.cc:2376] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5. Re-sending same reply.
I20260709 12:24:35.925837  1268 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.918622  1234 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:35.888468  1296 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.923759  1339 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.925941  1307 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:35.976052 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:35.926146  1311 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.930883   730 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:35.925355 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.927804  1168 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:35.985772  1168 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.987465  1168 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.929334  1290 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.931077  1274 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:35.938201  1198 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.952833  1312 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:35.944087  1346 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:24:36.014961  1312 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:36.015537  1346 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:36.015933  1312 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.983028 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:35.926790  1281 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.984361   709 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:36.022159   709 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:36.023066   709 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:35.964074  1234 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:36.027601 31691 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:36.029102  1296 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.018880  1203 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.031260  1154 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.020776  1358 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:24:35.995380  1243 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:35.996100 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:36.032866 30985 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.997694  1268 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:35.979765  1339 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.033721 31694 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:36.036082 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.015931  1346 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:35.998812   901 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:35.923657  1056 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:35.999236   730 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:35.993610  1242 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.038610 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.043546 31068 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.039494 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.039238  1346 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:36.041070 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 87345.6us
I20260709 12:24:36.041406 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.000415  1140 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:36.039819  1243 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:36.045122 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 161530us
I20260709 12:24:36.046329  1198 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:36.047397 30983 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:36.044447 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 166388us
I20260709 12:24:36.050699 30987 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:36.058136  1346 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:36.058730  1268 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:24:36.061628 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:36.064088  1308 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.837s	user 0.004s	sys 0.000s
I20260709 12:24:36.066579  1263 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:36.067030  1263 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.068647 31137 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:36.068898 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.069352 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.070690 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:36.073287 31137 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:36.073709 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 576) took 1382 ms. Trace:
I20260709 12:24:36.075695 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:36.076828 31135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1907) took 1479 ms (client timeout 1924 ms). Trace:
W20260709 12:24:36.077199 31135 rpcz_store.cc:269] 0709 12:24:34.597328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.710289 (+112961us) service_pool.cc:224] Handling call
0709 12:24:36.076658 (+1366369us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b5400063454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:36.076769 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653554560}
I20260709 12:24:36.075798 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:36.074067 31061 rpcz_store.cc:276] 0709 12:24:34.690854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.084327 (+393473us) service_pool.cc:224] Handling call
0709 12:24:35.084595 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.627076 (+542481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.073509 (+446433us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54000c5b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:36.073656 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246377216}
I20260709 12:24:36.081457 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:36.082036 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2027) took 1430 ms. Trace:
I20260709 12:24:36.082521 31137 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.084242 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.082578 31067 rpcz_store.cc:276] 0709 12:24:34.651825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.949812 (+297987us) service_pool.cc:224] Handling call
0709 12:24:34.969045 (+ 19233us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:34.972264 (+  3219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.972293 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:34.979481 (+  7188us) log.cc:844] Serialized 52 byte log entry
0709 12:24:35.113132 (+133651us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:35.244145 (+131013us) log.cc:844] Serialized 14 byte log entry
0709 12:24:35.350770 (+106625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.350844 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.063412 (+712568us) spinlock_profiling.cc:243] Waited 8.98 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:36.071717 (+  8305us) raft_consensus.cc:1685] finished
0709 12:24:36.081786 (+ 10069us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.081961 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4975,"prepare.queue_time_us":174119,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":8594,"raft.queue_time_us":92481,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":7971,"spinlock_wait_cycles":46960128,"thread_start_us":331379,"threads_started":3,"wal-append.queue_time_us":127339}
I20260709 12:24:36.021344  1221 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.092610 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:36.098376 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.099565 31068 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:36.101388 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:36.103850 31697 raft_consensus.cc:2368] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
I20260709 12:24:36.104941 31109 leader_election.cc:400] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4.
I20260709 12:24:36.106683 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.089993  1351 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:36.103607   864 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:24:36.108565 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.108623   864 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:36.109387  1246 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:36.078281  1281 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:36.110248  1246 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:36.110311  1290 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:36.109480   864 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:36.111054   864 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:36.112033   864 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.104187 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.109746 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:36.113572 31144 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:36.114159 31143 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:36.084560  1307 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:36.116452 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.117381 31697 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:36.088781  1198 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:36.088793  1187 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:36.118736 31144 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:36.123883 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.124054 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:36.085824  1358 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:36.093111  1277 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.056543  1140 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.128383 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.114256  1330 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:36.131680 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.139254 31696 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:36.080718  1296 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:24:36.133054 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:36.140826 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:36.141397 31139 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:36.089004   989 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:24:36.142127 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:36.142679 31139 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:36.129745 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:36.136318 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:36.126783 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.145246 31697 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.091445  1268 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:24:36.123749  1290 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:36.149299 31144 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:24:36.132826  1346 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:24:36.130656   730 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.170957 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.168430  1155 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.130115  1358 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:36.176477 31697 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:36.148517 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.216936 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:36.174762  1357 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:24:36.167066 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.178328  1353 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:36.124495  1198 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:36.185202 31695 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:36.165896  1140 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.145576  1351 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:36.140445 31689 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 12:24:36.181175 30990 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2006) took 3448 ms (client timeout 4410 ms). Trace:
W20260709 12:24:36.230373 30990 rpcz_store.cc:269] 0709 12:24:32.732973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.614524 (+881551us) service_pool.cc:224] Handling call
0709 12:24:36.180970 (+2566446us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400225e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:36.181116 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4034699008}
I20260709 12:24:36.177829   600 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.196111 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:36.173561  1263 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.196406 31137 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:36.146862   864 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.216869  1307 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:36.204347 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 112187us
I20260709 12:24:36.143623  1330 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:36.182771  1359 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:24:36.218878 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.218151 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:36.124017  1327 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:24:36.223140 31696 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:36.152357  1290 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:36.232277 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.241758 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.251214 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:36.234089  1154 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.237933 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:36.245770 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:36.227983 31691 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:36.142845  1243 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.218513 31144 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:36.198385  1356 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:24:36.253997 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.254107 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.238492  1056 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.257067 30986 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:36.257920 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:36.242636  1330 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:36.269644 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.273000 30989 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:36.259328 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.276540 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.252370  1290 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:36.276844 30992 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:36.277168 30984 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:36.268894  1346 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:36.280149 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.280824 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.282878 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.260354  1164 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:36.284777 30992 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:36.259030  1299 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:36.289649 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:36.308342  1359 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:36.274725  1242 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.279783  1363 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:24:36.252192  1358 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.291734  1365 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:24:36.291817  1311 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.294875 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 400) took 1225 ms. Trace:
I20260709 12:24:36.297132 31142 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:36.297860 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:36.300393 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:36.327983 31141 rpcz_store.cc:276] 0709 12:24:35.069235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.137247 (+ 68012us) service_pool.cc:224] Handling call
0709 12:24:35.139813 (+  2566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.383729 (+243916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.294656 (+910927us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540024a754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:36.294818 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550738816}
I20260709 12:24:36.329653 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.330276 30988 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:36.331542 30992 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:36.331687  1208 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:24:36.332450  1311 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.333189 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.335487  1278 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:36.335899  1278 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:36.336302  1278 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.336182  1234 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.338649 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.338824 31064 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:36.339226 31061 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.340144 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:36.341585 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.342370 31063 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:36.343336 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.343889 31063 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:36.260140  1154 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.347167 31059 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:36.347714  1154 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.349107 31059 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:36.351117 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.351094 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:36.338945 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:36.352463 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.310379 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.310328 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.259904  1198 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.304850  1221 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.308418  1373 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
W20260709 12:24:36.317720 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 70189.6us
I20260709 12:24:36.320031  1263 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.291934  1346 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:36.354398 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:36.291286 30985 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:36.369004 31063 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:36.380162 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.325228 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:36.324771  1164 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:36.382499 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:36.351507   600 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:36.387216 30988 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:36.392619 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:36.369494  1327 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.373128  1154 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:36.394901 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:36.396483 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:36.385308  1187 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.551817 31697 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
W20260709 12:24:36.552932 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.385568  1358 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:36.385221  1209 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:36.489079  1321 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:36.529385  1243 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.530051 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 224773us
I20260709 12:24:36.464233 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.556584  1243 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.482738 31061 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:36.583642 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.560160 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:36.391505  1198 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:36.541513   970 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.690s	user 0.007s	sys 0.004s
I20260709 12:24:36.587005 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:36.389971   600 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:36.377660   989 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:36.610813 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:36.625958 31139 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:36.627286 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:36.629071 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:36.636680 31137 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:36.637113 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:36.388986  1311 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.171s
I20260709 12:24:36.638518 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:36.639439 31067 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:36.624907 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:36.624221 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:36.640293 31137 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:36.643101 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1600) took 2793 ms. Trace:
W20260709 12:24:36.644603 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 227675us
I20260709 12:24:36.399049  1154 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:36.643458 31140 rpcz_store.cc:276] 0709 12:24:33.849394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.971031 (+1121637us) service_pool.cc:224] Handling call
0709 12:24:34.971713 (+   682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.220777 (+249064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.642880 (+1422103us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b540024a254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:36.643052 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571360128}
I20260709 12:24:36.641616 31141 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:36.640303 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.650744 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.651469 31067 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:36.392351  1354 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.226s
I20260709 12:24:36.418581 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:36.382465  1364 consensus_queue.cc:1175] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" responder_term: 9 status { last_received { term: 9 index: 7 } last_committed_idx: 6 last_received_current_leader { term: 9 index: 7 } }
I20260709 12:24:36.643939   970 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.684664 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:36.686194 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.618400  1363 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:36.556578  1321 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.693208 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.400238  1329 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.270s
I20260709 12:24:36.639851   730 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:36.687800 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:36.698561  1365 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:36.702556 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:36.545842  1278 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.648824  1140 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:36.721964 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.644918  1263 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:36.846877  1263 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.847293  1263 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.558341  1187 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.851243 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 712) took 1118 ms. Trace:
I20260709 12:24:36.851716 31695 rpcz_store.cc:276] 0709 12:24:35.732942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.871797 (+138855us) service_pool.cc:224] Handling call
0709 12:24:36.075563 (+203766us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:36.851047 (+775484us) spinlock_profiling.cc:243] Waited 98 ms on lock 0x7b540062ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:36.851173 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231014016}
I20260709 12:24:36.802237 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.856003 31061 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:36.857679 31061 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:36.862906 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 575) took 2172 ms. Trace:
I20260709 12:24:36.863294 31062 rpcz_store.cc:276] 0709 12:24:34.690611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.040333 (+349722us) service_pool.cc:224] Handling call
0709 12:24:35.040663 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.945504 (+904841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.862643 (+917139us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b5400384854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:36.862837 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043740032}
I20260709 12:24:36.865357 31062 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:36.871407   970 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.554476  1164 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.873927 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:36.877141 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:36.878603 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:36.803365 31142 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:36.653053  1154 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.442596  1346 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.649371   989 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.658958 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:36.808347 31067 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:36.829780 31694 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 12:24:36.830432  1345 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:36.902410  1345 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:36.903302  1345 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.904320  1345 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.906428 31060 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:36.908286 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:36.832332   864 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.588s	user 0.007s	sys 0.000s
I20260709 12:24:36.909219   953 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:36.909758   953 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:36.826428   627 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:36.910913   627 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:36.911345   627 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.911761  1339 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:36.841694  1290 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.001s
I20260709 12:24:36.914485  1290 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.915477  1056 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:36.915968  1056 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:36.916393  1290 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.695788 31692 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
W20260709 12:24:36.917014  1327 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.517s	user 0.005s	sys 0.001s
I20260709 12:24:36.917518  1327 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:36.916918  1056 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.918138 31063 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:36.919023  1243 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.843900  1321 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.660490   600 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.890465  1281 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:36.805632  1363 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.920526  1363 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.845458 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:36.923532 31068 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:36.924738  1140 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.925249  1140 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.881174 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:36.819466  1365 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:36.926625 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45554.7us
I20260709 12:24:36.927279  1140 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.927493  1353 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:36.927443 31661 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:36.928215  1353 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:36.929065  1353 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.930608  1347 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:36.931209  1347 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:36.932199   989 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.932351  1347 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.933169   989 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:36.874970   901 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:36.934557   901 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:36.927312  1365 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:36.936223  1365 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.936091   901 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:36.941094 31691 consensus_meta.cc:287] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.685s	user 0.008s	sys 0.008s
I20260709 12:24:36.819798 31059 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:36.879858 31061 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:24:36.894836  1379 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:36.896258  1278 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.901381  1263 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.910673   953 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:36.822772  1330 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.541s	user 0.019s	sys 0.000s
I20260709 12:24:36.938060   901 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:36.941685 31691 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
W20260709 12:24:36.954871 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45821.2us
I20260709 12:24:36.922685  1363 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:36.963308 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:36.942191  1378 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:24:36.949918  1056 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.951066  1290 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:36.951048  1327 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.972873  1380 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:36.967093  1330 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.985299  1327 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:36.997015  1123 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:36.997385  1327 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.997407  1330 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.999974  1263 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:36.982509  1290 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:36.818805 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.034296 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 213980us
W20260709 12:24:37.036387 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.039574 30989 raft_consensus.cc:2410] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:24:36.915107 31058 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:37.043221 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 406027us
I20260709 12:24:37.044612 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.047082 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.047960 31062 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:37.050169 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.053546 31065 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:36.923553 31067 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:36.943588 31693 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:37.058347 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:37.059684 31142 raft_consensus.cc:606] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:37.060338 31142 raft_consensus.cc:614] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:37.064106 31138 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:36.952858 31068 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:24:37.052471 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:37.071071  1203 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:37.037683 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:37.076222  1327 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.076973  1155 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:37.076993 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:37.082589  1330 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:37.083475  1387 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:37.084549  1382 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:37.088677  1228 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:37.057775 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.107241  1330 raft_consensus.cc:2764] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:37.108737 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.109346  1290 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.109848 31068 raft_consensus.cc:2410] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 12:24:37.110602 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73221.8us
I20260709 12:24:37.111758 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:37.112468 31067 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:37.112962  1363 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:37.113445  1363 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.113896  1363 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.114954  1317 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:37.117650  1317 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.105111 31142 raft_consensus.cc:606] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:37.118474 31142 raft_consensus.cc:614] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:37.118081  1317 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.119932  1198 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:37.103121  1228 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:37.122196  1198 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.132591  1198 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:37.123159 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 401) took 2048 ms. Trace:
I20260709 12:24:37.135082 31138 rpcz_store.cc:276] 0709 12:24:35.075075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.153352 (+ 78277us) service_pool.cc:224] Handling call
0709 12:24:35.153653 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.064785 (+1911132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.122804 (+ 58019us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b54000bead4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:37.123052 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4392023808}
W20260709 12:24:37.125620 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.133702  1198 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.138494  1198 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.139958  1363 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.155710  1281 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.156255  1281 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:37.156703  1363 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.142886  1228 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.157908  1228 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:37.151451 30955 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:37.154306 31067 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:24:37.132169 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.126739 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 163363us
I20260709 12:24:37.145915  1187 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.126505 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 802) took 1022 ms. Trace:
I20260709 12:24:37.157512  1281 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.170848  1298 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.183247  1298 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:37.184269  1353 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:37.185712  1353 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:37.176867 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.178062 31058 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:37.186504  1383 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:37.126585 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 800) took 1001 ms. Trace:
I20260709 12:24:37.176553 31693 rpcz_store.cc:276] 0709 12:24:36.104160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.231975 (+127815us) service_pool.cc:224] Handling call
0709 12:24:37.125908 (+893933us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54008273d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:37.126189 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1545397632}
I20260709 12:24:37.187603 31691 rpcz_store.cc:276] 0709 12:24:36.103599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.166821 (+ 63222us) service_pool.cc:224] Handling call
0709 12:24:37.103911 (+937090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.180570 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:37.190601  1363 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.190793  1299 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:37.191344  1363 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.192613  1299 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:37.193766  1363 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.194577  1384 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.195456  1384 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.195758  1380 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:37.196120  1155 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:37.197194  1311 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:37.197474   953 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:37.197959   953 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:37.199286 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:37.200201 30985 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:37.200662  1243 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:37.195830   981 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:37.201392  1281 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.201426  1243 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:37.203745  1243 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.205173  1243 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.181625 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:37.200613  1380 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:37.199111   627 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.205375 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.189996 30991 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:37.181984 31064 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:37.197839  1384 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:37.180837 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.199381   953 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.213307  1309 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:37.214082   953 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:37.214912 30988 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.215869 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:37.216176 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:37.217267 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:37.217448 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:37.206779  1380 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.214085  1309 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.219343 30991 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:37.219892  1380 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:37.220943 30991 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: [5.5-5.5]
W20260709 12:24:37.225313  1377 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 12:24:37.226266 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.227355 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.229779 30986 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:37.220968  1309 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:37.231201 30986 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:37.232143 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.197853  1311 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:37.233831 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 82518.1us
I20260709 12:24:37.234490  1384 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:37.234905  1384 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.235260  1384 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.237876 30985 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:37.236212  1298 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.240722  1155 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:37.242118 30986 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:37.227325 31135 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:37.247761   600 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:37.249830 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 197548us
I20260709 12:24:37.251236 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:37.251423 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.252342 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:37.245633 31064 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:37.244175 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.235260  1311 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.252467 31060 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:37.186163  1353 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:37.269428  1314 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:37.153977  1154 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.270390  1314 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.270844  1314 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:37.270941 31062 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.270898  1263 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:37.271859  1314 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.272420  1263 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.272974  1263 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.273053 30987 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:37.273231  1314 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.274803 31059 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:37.273192  1154 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:37.284045 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 103620us
I20260709 12:24:37.282374 31061 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:37.285982  1246 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:37.286595  1246 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.287071  1246 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:37.288772 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.289176 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.289710  1241 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:37.290323  1241 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:37.292356  1374 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.585s	user 0.002s	sys 0.000s
I20260709 12:24:37.292623  1241 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:37.293742 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.293802 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.294525 31061 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:37.295006  1225 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:37.282837 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:37.300093  1383 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:24:37.306490 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51108.8us
I20260709 12:24:37.306475 30987 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:24:37.309362  1154 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.309240  1307 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:24:37.310025  1154 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.312877 31063 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:37.312455 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:37.304271  1386 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:37.318261 30985 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.312580  1135 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.319525  1221 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:24:37.319774 31064 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.310201   627 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.312352  1154 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:37.310271  1307 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:37.335912  1307 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:37.336912  1307 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:37.330415 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.330588  1356 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:24:37.331043 30987 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:37.330969  1365 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:37.330612 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:37.332486 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.325012 30990 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:37.335304  1386 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:37.334167 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.310309   981 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:24:37.345468 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.345686  1383 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:37.346141  1383 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.345889  1353 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.346642 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:37.345703  1228 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:24:37.347440  1353 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.346545  1383 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.351859  1330 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.352037 31693 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:37.354646 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:37.355279  1364 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.355553 31138 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:37.355837  1364 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:37.354120 31698 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:37.350621 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.361071 31691 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:24:37.361593 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:37.362912  1228 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:37.362946 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.363415  1228 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:37.363822  1228 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:37.364068 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.367596 30988 raft_consensus.cc:2410] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:24:37.344081  1135 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.370253  1135 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.371277 31136 raft_consensus.cc:2410] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:37.348919  1307 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:37.374367 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1883) took 1133 ms. Trace:
I20260709 12:24:37.374760 30990 rpcz_store.cc:276] 0709 12:24:36.240993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.241116 (+   123us) service_pool.cc:224] Handling call
0709 12:24:37.374119 (+1133003us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54000249d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:37.374291 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1801694080}
I20260709 12:24:37.381095 30984 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:37.382112 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48114.7us
I20260709 12:24:37.382797 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.383050 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:37.381990  1155 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:37.348934 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.383544 31141 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:37.385481   627 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:37.385879   627 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:37.386391 31065 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.386730  1363 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.387027   627 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.387133  1363 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:37.388036  1299 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.388454  1299 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:37.388345  1363 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.390100  1363 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:37.391161 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 66673.5us
I20260709 12:24:37.343497  1386 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:37.394248  1386 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.395427 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.395646  1386 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:37.395938  1311 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:37.394757   981 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.396368  1311 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:37.399706 31060 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:24:37.400228 31060 pending_rounds.cc:85] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 2
I20260709 12:24:37.400457 31060 pending_rounds.cc:107] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:24:37.400746 31060 raft_consensus.cc:2889] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:37.399757 30987 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:24:37.416005 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.402235 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:37.426887  1168 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:37.427660 31063 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:37.427852 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:37.428869 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 102207us
I20260709 12:24:37.429075 31063 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.430615 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:37.431756 30987 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:37.432606 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.438690 31064 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:37.440277 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:37.399649  1311 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.449455  1311 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.450747 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.389760  1299 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.470477  1317 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.472000  1384 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.472992  1317 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.473999  1384 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.447440  1168 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.357052  1364 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:37.489630 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 148721us
W20260709 12:24:37.492064 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 109125us
W20260709 12:24:37.493263 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 53066.4us
I20260709 12:24:37.493680  1373 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:37.490439  1155 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.494244  1373 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:37.494417  1155 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.495393  1373 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.491561  1309 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:24:37.502799  1183 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.495683  1332 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.515622  1358 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.503021  1164 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:37.516083 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 85868.9us
I20260709 12:24:37.521047 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.505633   970 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:24:37.517475 30887 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:37.496017  1241 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:24:37.521126  1246 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.533470 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:37.534970 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.535053 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:37.577453 31138 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
W20260709 12:24:37.578720 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57244.3us
W20260709 12:24:37.581574 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.485385   981 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:24:37.503315  1155 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.523151  1299 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.588130 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.510461  1329 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:24:37.489254  1225 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 12:24:37.534591  1358 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:37.484381  1168 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.614261 31060 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:37.595374  1198 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:37.598408  1364 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.654474 30887 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:37.666054 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.672998 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.674638 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 70850.5us
I20260709 12:24:37.675438  1168 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:37.675940 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:37.676409 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 96228.4us
I20260709 12:24:37.677306 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:37.678411 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:37.684537 31061 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:37.692148 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1902) took 1388 ms. Trace:
I20260709 12:24:37.579964  1183 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.698817 30986 raft_consensus.cc:2410] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:37.698884   627 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.692529 30984 rpcz_store.cc:276] 0709 12:24:36.295602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.375412 (+ 79810us) service_pool.cc:224] Handling call
0709 12:24:37.379614 (+1004202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.381494 (+  1880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.683742 (+302248us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b54000249d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:37.683869 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2298994560}
W20260709 12:24:37.707628 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.705325 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 183409us
I20260709 12:24:37.717948 31067 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.684090  1327 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.683758   953 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:37.719861 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.727105 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:37.735127 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.697257  1203 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:37.729835 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:37.737237 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:37.742341 31065 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:37.744323 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.744880 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.738979   627 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:37.739086 30985 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:37.739771 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:37.730448  1298 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:37.756502 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 188795us
I20260709 12:24:37.757267  1346 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:24:37.758402  1234 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.761584  1228 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:37.751157  1399 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:24:37.752746 30889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 313) took 12733 ms (client timeout 15000 ms). Trace:
I20260709 12:24:37.753139  1345 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.754341  1241 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.762303  1123 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:37.750517  1056 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:37.762877 31698 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:37.763514  1246 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.767854  1309 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:37.769121  1183 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.769033  1358 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.835587  1358 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.836093 31065 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:37.837769   970 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:37.770303   981 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:37.838382   970 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.838631   981 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.840408   970 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:37.840683   981 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:37.709568  1383 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.843701  1383 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:37.770941   953 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.845552   953 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:37.846732 30990 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.773612  1225 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.847436  1225 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.849234  1225 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:37.811285  1241 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:37.775748  1380 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:37.776202 30983 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
W20260709 12:24:37.782567 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 169772us
I20260709 12:24:37.692001  1198 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.747263 30988 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:37.782850 30991 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:24:37.788578 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.773955 30887 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:37.790436  1221 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:37.790151  1263 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.517s	user 0.002s	sys 0.009s
W20260709 12:24:37.823868 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.827109  1309 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:37.825969  1410 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:24:37.823215  1164 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:24:37.829033 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 91875us
I20260709 12:24:37.797698  1168 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:37.797914 30889 rpcz_store.cc:269] 0709 12:24:24.799453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.810593 (+ 11140us) service_pool.cc:224] Handling call
0709 12:24:37.514503 (+12703910us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:29.078673 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:29.079367 (+   694us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:29.079413 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:24:29.079436 (+    23us) tablet.cc:662] Decoding operations
0709 12:24:29.081220 (+  1784us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:29.081338 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:29.081377 (+    39us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:24:29.083138 (+  1761us) tablet.cc:701] Row locks acquired
0709 12:24:29.083199 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:24:29.083394 (+   195us) write_op.cc:270] Start()
0709 12:24:29.083561 (+   167us) write_op.cc:276] Timestamp: P: 1783599869083318 usec, L: 0
0709 12:24:29.083589 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:24:29.084444 (+   855us) log.cc:844] Serialized 6659 byte log entry
0709 12:24:29.611582 (+527138us) op_driver.cc:464] REPLICATION: finished
0709 12:24:30.024740 (+413158us) write_op.cc:301] APPLY: starting
0709 12:24:30.024907 (+   167us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:30.025411 (+   504us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:30.025587 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:30.033249 (+  7662us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:30.372187 (+338938us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 12:24:30.380117 (+  7930us) write_op.cc:312] APPLY: finished
0709 12:24:30.520703 (+140586us) log.cc:844] Serialized 119 byte log entry
0709 12:24:30.574136 (+ 53433us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:30.581060 (+  6924us) write_op.cc:454] Released schema lock
0709 12:24:30.595476 (+ 14416us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:30.718052 (+122576us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:30.720104 (+  2052us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":335324,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":367176,"apply.run_cpu_time_us":10808,"apply.run_wall_time_us":667840,"num_ops":17,"prepare.queue_time_us":67933,"prepare.run_cpu_time_us":75735,"prepare.run_wall_time_us":183163,"raft.queue_time_us":85274,"raft.run_cpu_time_us":9753,"raft.run_wall_time_us":144309,"replication_time_us":527810,"thread_start_us":422069,"threads_started":4,"wal-append.queue_time_us":352253,"wal-append.run_cpu_time_us":13242,"wal-append.run_wall_time_us":2263043}]]}
I20260709 12:24:37.833384  1345 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:37.851341 30983 pending_rounds.cc:85] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 3
I20260709 12:24:37.869191 30983 pending_rounds.cc:107] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:24:37.869462 30983 raft_consensus.cc:2889] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:37.861364 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.889020 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:37.891064 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:37.897644 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.849848  1359 consensus_queue.cc:1175] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" responder_term: 7 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:37.903360 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.885349 30887 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 4 to 6, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 6 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:37.905274 31061 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 4: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:37.908027 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.908768 31060 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:37.890686  1353 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.869369  1263 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.919938 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.920571 31141 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
W20260709 12:24:37.921501 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.884465  1380 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.925784 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:37.926668 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:37.927971 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:37.971693 31063 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:37.896088  1168 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:37.990790 31058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.033s	sys 0.070s
W20260709 12:24:37.991436 31058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.033s	sys 0.070s
I20260709 12:24:37.971880 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:37.883088  1241 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:37.993716 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:37.999038 31138 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:24:37.998878 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 104699us
I20260709 12:24:38.002543   901 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:37.929266  1314 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:38.009434  1407 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.010092  1407 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.011229 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.011848   953 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.012145  1407 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:38.012410   953 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.014787  1406 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:38.015407  1406 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.017131  1314 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.019109  1314 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:38.019240  1380 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:38.012598  1410 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.003680 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 83847.8us
I20260709 12:24:37.998638  1363 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:38.027478 31691 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.037556  1168 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:38.032384 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.971109  1281 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:24:37.984714  1353 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:38.048923 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54455.7us
W20260709 12:24:38.049461 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.061235 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:38.063089 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:38.064201 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:38.065187 31691 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:38.066360 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.066936 31059 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:38.067664 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:38.068643 31143 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:38.069499   953 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:38.070186   953 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:38.071246 30986 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:38.071616 30986 pending_rounds.cc:85] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 5
I20260709 12:24:38.071847 30986 pending_rounds.cc:107] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:38.072052 30986 raft_consensus.cc:2889] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:38.073216  1410 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.075510  1410 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:38.076242  1345 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.076876 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.077469 31138 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:38.080687 31143 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:38.088279 30988 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:38.034951 30887 catalog_manager.cc:5697] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.097981  1056 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.102653 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.101356 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.103782 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.104204 31143 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 3.
I20260709 12:24:37.925671  1309 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.107191 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.108103 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:38.110562  1263 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.111403 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:38.111899  1307 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.737s	user 0.009s	sys 0.003s
I20260709 12:24:38.112524 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.113600 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.114323  1168 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:38.114873  1168 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.116890  1168 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.118690  1412 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:38.119154  1412 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:38.119529  1412 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.123111 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.124235 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.124476  1241 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.124874 30986 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:38.125802 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.129213 30990 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:24:38.129550 30990 pending_rounds.cc:85] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 5
I20260709 12:24:38.129796 30990 pending_rounds.cc:107] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:24:38.130528 30990 raft_consensus.cc:2889] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:38.132469 30992 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:38.132696 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1904) took 1836 ms. Trace:
I20260709 12:24:38.133112 30983 rpcz_store.cc:276] 0709 12:24:36.296090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.375832 (+ 79742us) service_pool.cc:224] Handling call
0709 12:24:36.394912 (+ 19080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.875195 (+1480283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.132435 (+257240us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540022e2d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:38.132640 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3167781632}
I20260709 12:24:38.133750 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1901) took 1838 ms. Trace:
I20260709 12:24:38.134141 30992 rpcz_store.cc:276] 0709 12:24:36.295354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.334041 (+ 38687us) service_pool.cc:224] Handling call
0709 12:24:36.351967 (+ 17926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.132874 (+1780907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.133516 (+   642us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b540014e6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:38.133698 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4093693824}
I20260709 12:24:38.135977  1353 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.136603  1345 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:38.136689  1353 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:38.137763  1345 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:38.104856  1228 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.139827 30986 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:38.143981  1383 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:38.106179 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.154083 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.154830 31138 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:38.163461 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:38.170660 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 106423us
W20260709 12:24:38.178964 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.179600 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.138136  1345 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:38.146503  1384 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.153892  1311 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.189795 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:37.996241  1164 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.186949 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:38.194792 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:38.195696 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.205286 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:38.208135 31062 raft_consensus.cc:2410] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:24:38.208571 31698 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:38.181573  1198 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.206153 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:38.211647 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:38.187458  1329 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.191730  1309 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.163046  1307 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:24:38.229713 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:38.218364 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:38.235582 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.239326 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:38.239995 31696 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.240356 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.241065 31063 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:38.241362 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:38.241971 31693 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.242360  1329 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.242784 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:38.244556  1329 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:38.245173 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:38.242312  1298 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:38.245949  1298 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:38.246052 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:38.246371  1358 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.246941 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:38.246835  1298 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.210975 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:38.248369  1298 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:38.248622 31691 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:38.211169  1123 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.249889  1123 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.250276  1198 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:38.210969  1404 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:38.257179 31136 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:38.214407  1281 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.262087 31141 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:38.266830 30887 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 5 to 6, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 6 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.269608  1263 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:38.222877  1353 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.204573  1384 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:38.272421 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 60846.9us
W20260709 12:24:38.274326 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.222815  1380 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:38.271740 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.278294 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:38.291033 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 72784.8us
I20260709 12:24:38.293179 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:38.251981  1123 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:38.271095  1409 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:38.260869  1404 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.214485  1410 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 12:24:38.270077  1263 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.261940  1327 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:38.264279  1183 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:38.309836 30984 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:38.246904  1358 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.314008 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:38.287797  1380 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:38.330201 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.359493 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.358594 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:38.362478 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:38.362686 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.287632  1314 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:38.305862  1406 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:38.406844 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 92967us
I20260709 12:24:38.387194 30887 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 3 to 7, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 7 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.409076 31696 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
W20260709 12:24:38.368239 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.410625 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.368911 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75827.4us
I20260709 12:24:38.372095  1421 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.372154 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.317384  1263 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:38.377737 31693 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:38.380596 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:38.381260 31135 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:38.381699 31141 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.382112 31136 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.386305  1418 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.170s
I20260709 12:24:38.307711  1246 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:38.314152  1183 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:38.363684  1380 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:38.301385  1353 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.314891  1404 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.298146  1281 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.364214  1410 consensus_queue.cc:1243] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:38.453987  1241 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:38.478967 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.453217  1123 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:38.295104  1345 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:38.479235 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.487154 30887 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.454798  1314 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.463902  1183 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:38.504499 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 256475us
I20260709 12:24:38.510746 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:38.524099  1402 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:38.535432  1411 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.115s
I20260709 12:24:38.539430  1412 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.541437  1412 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:38.555405  1345 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.523906  1314 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.533284 31690 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:38.564072  1353 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.566075  1431 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.566622  1431 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.569401  1431 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.569896  1431 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.570246 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:38.570986 31062 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:38.479938  1307 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.571848 30887 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 2, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 2 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.531512 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:38.577368 31698 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
W20260709 12:24:38.451344  1416 consensus_queue.cc:1175] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" responder_term: 5 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:38.591707 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.583254 31141 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:38.600051 31136 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
W20260709 12:24:38.618434 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 87004.8us
I20260709 12:24:38.591043  1358 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.586536 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.583689 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.628993 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:38.629945 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:38.630689 30983 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:38.611750  1425 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.065s	user 0.000s	sys 0.002s
I20260709 12:24:38.583333  1431 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.595270  1281 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.583990 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:38.616598  1228 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.632408 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.587728 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:38.596087  1384 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.643751 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.644294 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:38.652981 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 65338.8us
I20260709 12:24:38.631315  1234 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.641804  1123 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.611784  1307 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.701241 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:38.663017 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.653859 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:38.690133  1234 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.712882 30987 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:24:38.713279 30987 pending_rounds.cc:85] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 7
I20260709 12:24:38.713500 30987 pending_rounds.cc:107] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:24:38.713737 30987 raft_consensus.cc:2889] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:38.714793  1234 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:38.715447 30987 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:38.716710 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.657259  1281 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:38.721958  1281 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.650815  1359 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.722371  1281 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.694901  1431 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:38.696295  1183 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.726876  1183 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:38.727063 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:38.728479 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:38.728626  1307 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:38.653705  1228 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.729125  1380 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.730921  1228 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:38.732147  1384 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:38.732635  1384 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.733054  1384 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.734627  1364 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.736122  1364 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:38.736763  1364 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.711762 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.739064 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.712387 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:38.753703 31142 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.000s
I20260709 12:24:38.754236 31142 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:38.767823 30985 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:38.771870 30987 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:24:38.773777 31698 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:38.774039 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:38.772399 30987 pending_rounds.cc:85] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 5
I20260709 12:24:38.779876 30987 pending_rounds.cc:107] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:24:38.780123 30987 raft_consensus.cc:2889] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:38.782130 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.788188 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.789013 30983 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:38.765873  1246 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:38.789419 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.791204 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:38.792986 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:38.796551 31067 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:38.796576 31696 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:38.799751 31143 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:38.794412 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:38.775785  1434 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.803540 30985 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:38.784847  1234 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.787184  1431 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.785681  1416 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.816542 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.795450  1383 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:38.823778 31144 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:38.799466  1299 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.826917 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.824409 31061 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:24:38.795454  1412 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:38.827670 31141 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:38.784634  1314 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.803915  1363 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:38.874095 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:38.870895 30983 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:38.878701 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 368040us
W20260709 12:24:38.879361 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 403119us
I20260709 12:24:38.816670  1221 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.884555 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.804052  1155 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:24:38.815924  1364 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:38.892506 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:38.875736  1399 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:24:38.894351 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.901078 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:38.901407 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.901647 31067 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:38.899235 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.902566 31142 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:38.898435 31062 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:38.909750 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:38.911854 31064 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:38.881215 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:38.825301  1234 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:38.918787 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.919653 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.871129  1327 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:38.923887 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:38.871215  1299 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:38.911059  1309 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.689s	user 0.007s	sys 0.002s
I20260709 12:24:38.928243 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:38.827472  1431 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.904567   953 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:38.914505  1298 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.824156  1434 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.826918  1416 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.875598  1412 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:38.933743 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.936030 31141 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:38.926551  1164 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:38.940436  1329 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.924551 31690 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:39.004031 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.004717 31144 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
W20260709 12:24:39.006810 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.007633 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:38.960348  1281 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.009459 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:38.951205 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.011080  1373 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:39.011683  1373 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.012960  1431 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:38.958118  1432 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:24:39.027086 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 215029us
I20260709 12:24:39.027896 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:38.957547 31140 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:38.957238  1299 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:38.904651  1363 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.933672  1442 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:38.950907  1268 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:38.930815   970 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:38.995265  1407 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:39.050899  1314 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:39.051539 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.052398 31698 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:38.996006 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:39.052903 30983 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.3-6.3]
W20260709 12:24:39.053503 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 57575.1us
I20260709 12:24:38.997443 31692 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:39.034492  1164 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.055083   981 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.055205 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.055527   981 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.055923   981 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.056288  1164 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:39.057212 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:39.057767 31067 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.058974  1299 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:39.059653  1299 consensus_queue.cc:1243] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:39.023056  1329 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:38.999554 31689 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:39.002483  1427 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 12:24:39.002619 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.061657 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:39.003325 31141 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:38.941402  1221 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:39.004076 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 97337.1us
I20260709 12:24:39.061978  1329 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.063838 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.062237 30987 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.064432 30983 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:39.067963 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:39.068941 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:38.999109  1412 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:38.951723  1358 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:39.033994 31695 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:38.930640 31691 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
W20260709 12:24:39.014647  1414 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:24:39.100286 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:39.044981 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:39.045557 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.101610 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.023578  1383 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:39.049038 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:39.050209   970 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.107430 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:39.050943  1407 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:39.063709 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:39.119666 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:39.120137 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:39.120231 31061 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:39.120990 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:39.122473 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:38.952786  1168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.007s	sys 0.021s
I20260709 12:24:39.069010 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:39.139348 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 70502.1us
I20260709 12:24:39.062436 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.011089  1281 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.102737 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:39.013883  1443 raft_consensus.cc:2764] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:24:39.039443  1418 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.042990  1380 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:38.960414  1298 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.110579 31064 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:39.095399  1241 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:39.063555 31693 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:38.953511   953 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:39.067492  1435 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 12:24:39.072911  1451 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.135564 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:39.141892 31138 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.145195 31062 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:39.141454 31694 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:39.122051  1431 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:39.160316 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 91375.7us
I20260709 12:24:39.163268 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:39.170825 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 107205us
I20260709 12:24:39.151163 31692 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:39.116466  1383 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.162485 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.174981 31061 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:39.077121  1442 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.134732  1438 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
W20260709 12:24:39.150596  1168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.011s	sys 0.021s
I20260709 12:24:39.196019 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:39.196640 31693 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:39.199045 30987 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:39.176096 31067 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:39.151298  1373 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.062670  1221 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.157070  1411 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:39.182898 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:39.108129  1268 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.164189  1241 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:39.183599 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48123.6us
W20260709 12:24:39.200241  1263 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.730s	user 0.006s	sys 0.003s
I20260709 12:24:39.158193  1447 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.220329 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.191830 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:39.201407  1440 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 12:24:39.217093 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:39.223145 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.192826  1329 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.183156  1458 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.201256 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.229498 30990 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:39.219429  1241 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.234781  1441 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260709 12:24:39.248653  1459 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:24:39.231389  1234 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.249643 31691 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:39.175993  1451 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.251187 31691 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:39.251940  1451 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:39.221472 31689 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:39.177059  1431 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.257736 30986 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:39.167811  1418 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.146646  1327 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.236500 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:39.232465  1458 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.225600 30987 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:39.224213 30992 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:39.236639 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:39.255206 31058 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:39.261025 31138 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
W20260709 12:24:39.262518  1456 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:24:39.263821  1268 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:39.264283  1444 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:24:39.233018  1299 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.265326 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.193110  1407 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.225870  1263 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.254913  1434 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.274788  1299 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.274842  1450 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.278885   953 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:39.292529  1431 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.269655 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:39.292829 31060 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:39.292529  1263 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:39.293736 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:39.269580 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:39.269851  1384 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.000s
I20260709 12:24:39.268128  1416 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:39.265995  1363 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:39.293965  1298 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.293730  1383 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.296277 31692 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:39.296823  1298 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:39.297793  1434 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.298261  1221 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.295197  1407 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.296334  1450 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.301149  1450 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:39.303061  1450 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:39.304852 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:39.305891  1450 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:39.287323  1451 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:39.307241 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.307497 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:39.270810 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:39.309795 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73275.1us
I20260709 12:24:39.309944 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:39.273162 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.264846  1452 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:39.289562  1404 consensus_meta.cc:287] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.793s	user 0.009s	sys 0.001s
I20260709 12:24:39.268472  1458 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:39.333402  1454 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:39.292099  1299 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:39.308799   981 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.334394  1438 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:39.334831  1431 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.335243  1431 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.335579  1431 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.336149   981 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:39.310734 31698 raft_consensus.cc:2410] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:39.326542 30986 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:39.311205  1384 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.339493 30986 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:39.340682 31693 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:39.340876 30989 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:39.341238  1168 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:39.328424 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:39.342108 31142 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:39.291000  1442 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:39.345562 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:39.356676  1384 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:39.357702  1438 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:39.358132  1438 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.358489  1438 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.323062  1452 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.311022  1450 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.374073 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.375090 31139 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:39.346992  1443 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:39.377683 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 155156us
I20260709 12:24:39.362668  1228 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.380143 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:39.338092  1404 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.373401  1452 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.386629 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.387436 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:39.381799  1228 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.395964 31138 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:39.396592 31136 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:39.397321 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:39.403224  1450 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.403724  1450 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.404109  1450 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.407465  1458 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.409600  1458 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:39.382656  1443 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.411922  1443 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:39.412351 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 384508us
I20260709 12:24:39.413079 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:39.413503  1450 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.414105 30958 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:39.415040 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:39.416407 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:39.420420 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.418695 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:39.398521  1228 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.396838 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:39.415668  1450 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.424489  1443 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:24:39.438407  1241 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.435786  1155 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:39.440289  1155 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.440663  1241 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:39.440811  1155 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:39.436792  1462 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.438705 31693 raft_consensus.cc:2376] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4. Re-sending same reply.
I20260709 12:24:39.442118 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:39.436301 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.427134 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:39.444721 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:39.397039  1327 raft_consensus.cc:2764] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:39.416254  1246 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.454753 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:39.456990 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:39.447619 31698 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:39.427723  1364 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:39.427879  1470 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:39.456151 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:39.427769  1359 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:39.463764 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:39.425191  1475 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.469986 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:39.458309 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 78324.7us
I20260709 12:24:39.470820 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.471166 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:39.471462 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.472189 31690 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:39.467105 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.473486 31692 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:39.475797  1383 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.476295  1383 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.477663 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:39.478945  1461 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.468s	user 0.004s	sys 0.000s
I20260709 12:24:39.481294  1452 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.487043 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:39.471462 30984 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:39.473944  1446 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.491927 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.500128 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.500919 30983 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:39.502493 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:39.503309 31065 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:39.460927  1407 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:39.490805 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:39.507772 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.508622 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:39.512481 31063 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.466957  1234 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.517297 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:39.519938 31058 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.466504  1443 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:39.469282  1416 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.467273  1470 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:39.445973   953 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:24:39.516675  1359 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:39.489944  1383 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.525846 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:39.536439 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:39.536720 30983 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.485179  1475 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.487843  1241 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.529807 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:39.525708 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:39.489996  1446 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.500170  1327 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:39.501900  1412 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:39.469053  1299 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:24:39.469192  1307 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:24:39.540390 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:39.571619 31696 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:39.529119  1443 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:39.539440  1475 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.528533  1416 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.535820  1470 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:39.539009  1241 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:39.535121  1442 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:39.542088  1446 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.561936  1383 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:39.562777 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:39.595980 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:39.563030 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.563467 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:39.564553  1449 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.561s	user 0.001s	sys 0.002s
I20260709 12:24:39.507375  1364 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:39.523393  1447 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:39.541945 31143 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:39.567728 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:39.569621 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 78891.3us
I20260709 12:24:39.569945 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.570761 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:39.572373  1472 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.572892 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.573704 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.574558 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:39.648623 31138 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:39.575017  1412 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:39.649837  1412 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.575444  1484 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:39.577297  1479 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 12:24:39.579289  1452 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:39.592936 30983 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:39.593752 31058 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:39.597118 31064 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:39.529312  1234 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.588585  1416 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.637004  1359 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.637971 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2070) took 1225 ms. Trace:
I20260709 12:24:39.645606  1228 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.638345  1364 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.639101   981 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:39.644726  1446 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.682982 30988 rpcz_store.cc:276] 0709 12:24:38.412486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.412637 (+   151us) service_pool.cc:224] Handling call
0709 12:24:38.419653 (+  7016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.453542 (+ 33889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.453576 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:38.463006 (+  9430us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.871493 (+408487us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:38.871530 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.871595 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.564015 (+692420us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:39.637644 (+ 73629us) raft_consensus.cc:1685] finished
0709 12:24:39.637668 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.637904 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4351,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":13654,"spinlock_wait_cycles":25875712,"thread_start_us":298857,"threads_started":1,"wal-append.queue_time_us":511667}
I20260709 12:24:39.660653  1353 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.656064  1447 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:39.656323 31063 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
W20260709 12:24:39.657042  1430 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.654s	user 0.002s	sys 0.001s
I20260709 12:24:39.656890 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.657850 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:39.830000 31140 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:39.830161 30992 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.594553  1241 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.652668  1434 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.636691 30986 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:39.591786  1327 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:39.589298  1443 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:39.671765 31061 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:39.662002 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:39.674147 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:39.843211 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 169165us
I20260709 12:24:39.844499 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:39.690937 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:39.845408 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 154583us
I20260709 12:24:39.646974 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:39.659929  1484 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.602439  1442 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:39.812947 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:39.832309  1474 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.661163  1431 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.850618 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:39.851203 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2057) took 1780 ms. Trace:
I20260709 12:24:39.854656 31693 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:39.860023  1327 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.832763  1353 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.848289 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:39.837587  1434 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:39.835122  1479 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 12:24:39.897541 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 482540us
W20260709 12:24:39.868904 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 207007us
I20260709 12:24:39.898684 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:39.899480 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:39.900489 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:39.871457 31058 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:39.847656 31135 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:39.883013   953 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:39.893180 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44996.1us
I20260709 12:24:39.698421   981 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:39.902902 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:39.869918  1268 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.901641 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:39.890302  1438 consensus_meta.cc:287] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
I20260709 12:24:39.907061 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:39.907701 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:39.907920 31694 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:39.867136  1484 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.884609  1246 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.879410  1168 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:39.958487 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:39.959801 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.885243 30991 rpcz_store.cc:276] 0709 12:24:38.070317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.071041 (+   724us) service_pool.cc:224] Handling call
0709 12:24:38.071338 (+   297us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:38.103124 (+ 31786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.103152 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:38.105821 (+  2669us) log.cc:844] Serialized 52 byte log entry
0709 12:24:38.649309 (+543488us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:38.940194 (+290885us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.955270 (+ 15076us) log.cc:844] Serialized 14 byte log entry
0709 12:24:39.002843 (+ 47573us) spinlock_profiling.cc:243] Waited 9.8 ms on lock 0x7b180040d500. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:24:39.405277 (+402434us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b180040d500. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:24:39.405449 (+   172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.405520 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.656413 (+250893us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:39.851004 (+194591us) raft_consensus.cc:1685] finished
0709 12:24:39.851023 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.851140 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122150,"prepare.queue_time_us":194893,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":529678,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":273,"spinlock_wait_cycles":105805952,"thread_start_us":694971,"threads_started":2,"wal-append.queue_time_us":721775}
I20260709 12:24:39.903149   953 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.970252   981 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.971506   981 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.970523  1463 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:39.905375  1268 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.905994  1307 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:39.907733  1411 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:39.905181  1472 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.908610  1407 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:39.961651  1481 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:24:39.936609 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:39.990084 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:39.994942 31135 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:39.894479  1155 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:39.967336  1198 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:39.974855  1168 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:39.974639   953 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.986788  1268 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:40.011679 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.993140  1472 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:40.058005 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 467323us
W20260709 12:24:40.059320 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 155174us
I20260709 12:24:40.060016  1451 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:40.060403  1263 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:39.987406 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:39.997761  1446 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:40.102411  1234 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:40.104065  1486 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:40.104615  1486 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:40.105695  1487 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:40.105852 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.106158  1487 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.106896 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.131430 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:40.131738 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:40.132043 30985 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:40.110138 31135 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:40.117830 30992 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:40.122720  1451 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:40.126598 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:40.129058 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:40.002192  1463 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.130352 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:40.151475 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1643) took 1147 ms. Trace:
I20260709 12:24:39.928654  1438 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.062060  1458 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.065361  1472 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:40.144219 31137 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:39.891995  1474 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.146179 30984 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:40.058950  1246 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.129565 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:40.062189  1198 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.142738  1487 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:40.135373 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:40.155597 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.130062  1412 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.151877 31142 rpcz_store.cc:276] 0709 12:24:39.000061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.002310 (+  2249us) service_pool.cc:224] Handling call
0709 12:24:40.147729 (+1145419us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b540001b154. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:40.147882 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643417344}
I20260709 12:24:40.148305 31141 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:40.051659  1307 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:24:40.055627  1442 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:39.981587  1447 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.151981  1493 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.199s
I20260709 12:24:40.105438  1486 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.320668 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.484628 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.484990 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:40.485443 31135 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:40.485525 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:40.444666 31059 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:40.414842  1494 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:24:40.487738 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 446) took 1007 ms. Trace:
I20260709 12:24:40.199257  1411 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:40.488788  1411 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.488175 31139 rpcz_store.cc:276] 0709 12:24:39.480609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.491932 (+ 11323us) service_pool.cc:224] Handling call
0709 12:24:39.526825 (+ 34893us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.566983 (+ 40158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.567012 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.571714 (+  4702us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.862437 (+290723us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:39.862475 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.862538 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.487556 (+625018us) raft_consensus.cc:1685] finished
0709 12:24:40.487576 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.487687 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31128,"raft.queue_time_us":28511,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":5392,"thread_start_us":254586,"threads_started":1,"wal-append.queue_time_us":548502}
I20260709 12:24:40.422019 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.430655  1431 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:40.490780 31138 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:40.357422  1438 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:40.429677  1470 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.821s	user 0.005s	sys 0.000s
I20260709 12:24:40.286473  1263 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:40.492015  1470 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.494508  1470 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:40.496007  1493 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.161661  1168 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.496582  1493 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.498600  1493 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:40.499119  1493 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:40.499473  1493 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.499845  1493 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:40.442939 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 307637us
I20260709 12:24:40.501457 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:40.161496  1458 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.152979  1228 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:40.483466 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 351779us
I20260709 12:24:40.524633 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:40.532629 31058 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:40.538093 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:40.320585  1452 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.545091 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:40.481424  1451 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:40.153105  1358 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.382s
I20260709 12:24:40.507649  1474 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:40.481876  1298 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:40.482323  1463 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:40.416561  1404 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:40.533635  1407 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:24:40.561194  1498 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:24:40.563781 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.124s	sys 0.224s
I20260709 12:24:40.324851  1246 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:40.564162 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.124s	sys 0.224s
W20260709 12:24:40.574368 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 513847us
I20260709 12:24:40.370877  1484 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.552706   901 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:40.509970  1412 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.578639 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:40.581851 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:40.564000  1155 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:40.585356 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:40.585165 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 495) took 1019 ms. Trace:
I20260709 12:24:40.588093 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:40.548426  1452 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.575294  1463 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:40.588518 30988 rpcz_store.cc:276] 0709 12:24:39.565645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.699057 (+133412us) service_pool.cc:224] Handling call
0709 12:24:39.833153 (+134096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.870317 (+ 37164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.870364 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.881427 (+ 11063us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.061851 (+180424us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.108584 (+ 46733us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.584772 (+476188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.584904 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.584946 (+    42us) raft_consensus.cc:1685] finished
0709 12:24:40.584959 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.585105 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34491,"prepare.queue_time_us":66901,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":202128,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":17262,"spinlock_wait_cycles":32531840,"thread_start_us":574338,"threads_started":2,"wal-append.queue_time_us":291331}
I20260709 12:24:40.563277  1451 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:40.595461 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 110520us
I20260709 12:24:40.563328  1298 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.586827   901 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:40.586892  1234 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:40.587395  1507 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:40.603577 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:40.587080  1474 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:40.592229  1164 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:40.590900  1470 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:40.606853 30984 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:40.608903 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1793) took 1036 ms. Trace:
I20260709 12:24:40.614296 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:40.610980 31063 rpcz_store.cc:276] 0709 12:24:39.572819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.832278 (+259459us) service_pool.cc:224] Handling call
0709 12:24:39.832668 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.832878 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.832901 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.840600 (+  7699us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.994469 (+153869us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:39.994905 (+   436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.994977 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.608707 (+613730us) raft_consensus.cc:1685] finished
0709 12:24:40.608727 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.608845 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":143037,"threads_started":1,"wal-append.queue_time_us":306629}
I20260709 12:24:40.616472 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:40.617066 30988 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:40.615797 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:40.600199  1155 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:40.623487 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 122095us
I20260709 12:24:40.625422 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:40.612164  1443 consensus_meta.cc:287] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.722s	user 0.002s	sys 0.007s
I20260709 12:24:40.622747 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:40.628147 31059 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:40.615203 31109 connection.cc:256] RPC call timeout handler was delayed by -1.0522042239999791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
I20260709 12:24:40.606577  1507 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:40.629205 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:40.628638 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90619.1us
I20260709 12:24:40.630281 31061 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:40.599038  1493 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.634418 31058 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:40.597472  1463 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.637787 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:40.611402  1470 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:40.618654  1164 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:40.647184  1512 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:40.647083   901 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.648794  1452 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:40.650584 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:40.677740 31063 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:24:40.652954  1500 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:24:40.665947  1515 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.667790  1493 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:40.669973 30992 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:40.622648  1198 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:40.678653 30988 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:40.684615 30955 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:40.698022 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:40.698088 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:40.699846  1472 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.501s	user 0.003s	sys 0.006s
I20260709 12:24:40.700248  1472 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.702373  1472 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:40.703896 31691 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:24:40.704053  1472 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:40.704536  1472 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:40.705428 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:40.705408  1472 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.640944  1507 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:40.640614  1458 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:40.715596 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 170566us
I20260709 12:24:40.607849  1407 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.671344  1470 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:40.765113 31689 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 12:24:40.767177  1514 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:24:40.693833  1503 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:24:40.695966  1516 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 12:24:40.706722  1515 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.737948 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:40.677591  1228 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:40.630200  1443 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.738322  1358 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:24:40.744637 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:40.676110  1512 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:40.695133  1383 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.716919  1155 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:40.738524  1511 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:40.794479 31064 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:40.745124 31691 raft_consensus.cc:2368] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
W20260709 12:24:40.817981 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 192609us
I20260709 12:24:40.796850 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:40.800360 31698 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:40.789816 31063 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:24:40.863915 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 45116.5us
I20260709 12:24:40.865813 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:40.871842 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:40.872188 31135 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:40.872463 31138 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:40.874190 30984 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:40.778187  1241 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.900s	user 0.002s	sys 0.007s
W20260709 12:24:40.780261  1416 consensus_meta.cc:287] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.951s	user 0.008s	sys 0.000s
I20260709 12:24:40.877916 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.878232 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:40.794932  1504 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:24:40.883792 31135 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:40.800758  1383 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.784119  1451 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:40.879937 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:40.880373 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.880635 31059 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
W20260709 12:24:40.806033  1434 consensus_meta.cc:287] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.877s	user 0.004s	sys 0.004s
I20260709 12:24:40.880963 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.785118  1263 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:40.817762  1452 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.890592 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:40.890750 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:40.882457  1526 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.889588 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:40.894068 31138 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:24:40.807467  1246 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:40.806495  1511 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:40.798725  1412 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:40.786806  1327 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:40.800529  1512 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:40.807592  1517 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:40.915850 31059 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:40.802999  1298 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:40.932041 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 234080us
I20260709 12:24:40.873236  1358 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:24:40.936965 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 46329.2us
I20260709 12:24:40.940833 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:40.941660 30991 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
W20260709 12:24:40.928699 31664 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.812s (SENT) [suppressed 34 similar messages]
W20260709 12:24:40.915836 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.807269  1463 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:40.792927  1443 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:40.806162  1155 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.962522 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:40.927007  1526 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.918708  1484 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:40.966481 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:40.952448 31063 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:40.968173 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:40.801937  1487 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.924620  1511 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:40.784983  1164 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.932673  1246 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:40.975417 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 87107.6us
I20260709 12:24:40.908619  1198 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:24:40.894164  1416 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.886060  1241 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:40.918360  1434 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:40.986523 31060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2087) took 1877 ms (client timeout 2067 ms). Trace:
W20260709 12:24:40.983682 31136 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 434) took 1911 ms (client timeout 1812 ms). Trace:
I20260709 12:24:40.983716 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 493) took 1418 ms. Trace:
I20260709 12:24:40.985154 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:40.938401  1327 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:40.981565 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:41.003345 31137 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:41.004256 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:41.005503 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:41.006153 31137 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:41.007570 31137 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:40.989701 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:40.988714 30990 rpcz_store.cc:276] 0709 12:24:39.565070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.659332 (+ 94262us) service_pool.cc:224] Handling call
0709 12:24:39.683399 (+ 24067us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.872184 (+188785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.872230 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.872781 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.192019 (+319238us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:40.192080 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.423207 (+231127us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:40.423267 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.974190 (+550923us) raft_consensus.cc:1685] finished
0709 12:24:40.974232 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.983661 (+  9429us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":946,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":7554,"spinlock_wait_cycles":359156608,"thread_start_us":255517,"threads_started":1,"wal-append.queue_time_us":657452}
I20260709 12:24:40.992820 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:40.994401 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:40.986953 31060 rpcz_store.cc:269] 0709 12:24:39.103600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.105243 (+  1643us) service_pool.cc:224] Handling call
0709 12:24:40.981297 (+1876054us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400383954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:40.981406 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393535232}
I20260709 12:24:40.923435  1452 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:41.011219 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:41.012104 31060 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:41.012404  1452 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:40.987931 31136 rpcz_store.cc:269] 0709 12:24:39.069766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.069886 (+   120us) service_pool.cc:224] Handling call
0709 12:24:40.980958 (+1911072us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b540024a754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:40.981102 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676447616}
I20260709 12:24:41.012998 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:41.013746 31060 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:41.018707 31068 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:41.009939 31063 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:40.968458  1155 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:41.002131 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:41.024469 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 134993us
I20260709 12:24:41.026023 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:40.974749  1525 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:41.028828 31143 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:40.944797  1358 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:40.984043  1263 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:41.056360  1522 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 12:24:41.058203 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 118452us
W20260709 12:24:41.040663 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 111975us
I20260709 12:24:41.050036  1383 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:41.050446  1443 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:41.025487  1246 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:41.054288  1521 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:41.055137   953 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.960s	user 0.009s	sys 0.001s
I20260709 12:24:41.020284  1364 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:41.059290 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:41.190562 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 131339us
I20260709 12:24:41.031903  1531 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.257565  1416 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:41.258315  1405 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:41.161762  1383 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:41.022212  1447 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:41.055032 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:41.161456 31689 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:41.323405 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:41.360672 31690 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:41.361935 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:41.165484  1525 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:41.056396  1263 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:41.363899 31690 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:41.386732  1501 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 12:24:41.389492 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 494) took 1824 ms. Trace:
I20260709 12:24:41.260622  1364 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:41.389978 30983 rpcz_store.cc:276] 0709 12:24:39.565361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.682740 (+117379us) service_pool.cc:224] Handling call
0709 12:24:39.683105 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:39.688603 (+  5498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.688649 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:39.689348 (+   699us) log.cc:844] Serialized 52 byte log entry
0709 12:24:40.096675 (+407327us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:40.549066 (+452391us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.550973 (+  1907us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.739479 (+188506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.739578 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.742804 (+  3226us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b180040cd20. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:24:41.389230 (+646426us) raft_consensus.cc:1685] finished
0709 12:24:41.389260 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.389424 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32270,"prepare.queue_time_us":65009,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":80392,"raft.queue_time_us":83679,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":143105,"spinlock_wait_cycles":20745088,"thread_start_us":774015,"threads_started":3,"wal-append.queue_time_us":505883}
I20260709 12:24:41.064124 31135 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:41.194368   953 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:41.446118 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 420142us
I20260709 12:24:41.468890  1530 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:41.469573  1530 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:41.470012  1530 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:41.473676  1533 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.401s	user 0.000s	sys 0.000s
I20260709 12:24:41.474931  1526 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:41.483711  1534 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
W20260709 12:24:41.494478  1529 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:24:41.475749  1246 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:41.364424  1447 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:41.508682  1470 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.000s
W20260709 12:24:41.541138  1508 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.001s	sys 0.012s
W20260709 12:24:41.541584  1508 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.001s	sys 0.012s
W20260709 12:24:41.548249 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 357000us
W20260709 12:24:41.511596  1507 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.720s	user 0.009s	sys 0.000s
W20260709 12:24:41.540411  1493 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.003s	sys 0.009s
I20260709 12:24:41.267310  1531 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:41.365222  1327 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:41.560156 31698 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.031s	sys 0.028s
W20260709 12:24:42.106956 31698 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.032s	sys 0.052s
W20260709 12:24:41.690104 30988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.012s	sys 0.007s
W20260709 12:24:42.107962 30988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.104s	user 0.018s	sys 0.011s
W20260709 12:24:42.108085 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 1.11842e+06us
I20260709 12:24:42.108883 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2099) took 1169 ms. Trace:
W20260709 12:24:42.107023 31060 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.092s	user 0.029s	sys 0.072s
I20260709 12:24:42.114234 31060 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:42.114916 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2102) took 1181 ms. Trace:
I20260709 12:24:42.115309 31060 rpcz_store.cc:276] 0709 12:24:40.933765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.012713 (+ 78948us) service_pool.cc:224] Handling call
0709 12:24:42.114853 (+1102140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.121114 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:42.109339 30988 rpcz_store.cc:276] 0709 12:24:40.939853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.940405 (+   552us) service_pool.cc:224] Handling call
0709 12:24:40.967723 (+ 27318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.967952 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:40.984798 (+ 16846us) log.cc:844] Serialized 14 byte log entry
0709 12:24:42.108340 (+1123542us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.108408 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:42.108465 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.108633 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.108813 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18711,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":174352,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":128387,"thread_start_us":1123170,"threads_started":2,"wal-append.queue_time_us":61243,"wal-append.run_cpu_time_us":1085,"wal-append.run_wall_time_us":1001118}
I20260709 12:24:42.129933 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:42.131054 31135 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.134426 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:42.134501  1550 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.135214  1550 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.135697 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:42.136761 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:42.136950 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2096) took 1223 ms. Trace:
I20260709 12:24:42.137251 31137 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:42.137739  1550 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:42.138273 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:42.138895 30983 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.143646 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:42.145183 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:42.137310 31060 rpcz_store.cc:276] 0709 12:24:40.914157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.136226 (+1222069us) service_pool.cc:224] Handling call
0709 12:24:42.136495 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.136655 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.136680 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.136718 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:42.136747 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.136798 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.136900 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.119861 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:42.174787 30955 connection.cc:256] RPC call timeout handler was delayed by -1.0580608489999577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 12:24:41.630486  1475 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 2.032s	user 0.009s	sys 0.004s
W20260709 12:24:41.628048  1268 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.528s	user 0.006s	sys 0.011s
W20260709 12:24:42.088915  1383 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.740s	user 0.008s	sys 0.003s
I20260709 12:24:42.249809 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:42.250561 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:41.877595  1405 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.002s	sys 0.019s
W20260709 12:24:42.123948  1493 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.178s	user 0.005s	sys 0.012s
W20260709 12:24:41.961683  1155 consensus_meta.cc:287] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.911s	user 0.006s	sys 0.000s
I20260709 12:24:42.262889 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:42.318902 30990 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.264605 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2100) took 1338 ms. Trace:
I20260709 12:24:42.320065 31060 rpcz_store.cc:276] 0709 12:24:40.926477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.149910 (+1223433us) service_pool.cc:224] Handling call
0709 12:24:42.263989 (+114079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.264205 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.264241 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.264296 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:42.264340 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.264407 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.264541 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:42.280920 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 147160us
I20260709 12:24:42.323597 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:42.324277 31696 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.296237 30983 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:42.297333 31135 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:41.625700  1241 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.120669  1358 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:42.108604  1463 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:41.699474  1536 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:41.539734  1470 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:42.251333  1508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.012s
I20260709 12:24:42.327579 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:42.265314  1405 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.004s	sys 0.019s
I20260709 12:24:42.329206 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:42.410104 30988 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:42.330956 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:42.411499 30991 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:42.244055   970 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.244841  1550 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.352437  1418 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.328900  1545 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:42.360394 31693 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 454) took 2821 ms (client timeout 2437 ms). Trace:
I20260709 12:24:42.363179 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:42.435736 31060 raft_consensus.cc:2410] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:24:42.328683  1383 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:42.361014  1531 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.361680  1447 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.293833  1475 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.406729 30983 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
W20260709 12:24:42.348953  1511 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.361s	user 0.009s	sys 0.003s
I20260709 12:24:41.620707  1525 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.358209 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:42.483160 31136 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:42.439739 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2095) took 1509 ms. Trace:
W20260709 12:24:42.405946  1508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.006s	sys 0.012s
W20260709 12:24:42.484706 30992 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2094) took 1871 ms (client timeout 1690 ms). Trace:
I20260709 12:24:41.689044  1364 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.442301 31696 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:42.293036  1268 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.516978  1268 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:42.518546 30987 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:42.460270 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:42.520299 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 60127.5us
I20260709 12:24:42.521643 30958 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:42.343411  1155 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.244366  1507 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.418465   953 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.415838  1405 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:42.430370  1536 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:42.485107 30992 rpcz_store.cc:269] 0709 12:24:40.613618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.613948 (+   330us) service_pool.cc:224] Handling call
0709 12:24:42.484520 (+1870572us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b54002c2254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:42.484642 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126071680}
W20260709 12:24:42.513932  1512 consensus_meta.cc:287] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.575s	user 0.006s	sys 0.004s
I20260709 12:24:42.514460 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 828) took 3062 ms. Trace:
I20260709 12:24:42.514662  1383 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:41.971688  1263 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.484098 31068 rpcz_store.cc:276] 0709 12:24:40.913691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.024869 (+111178us) service_pool.cc:224] Handling call
0709 12:24:41.045823 (+ 20954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.424836 (+379013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.424879 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:41.424942 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:41.424987 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.422957 (+997970us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54001bb554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:42.423017 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.423181 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871143168}
I20260709 12:24:42.373591  1463 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.461841  1418 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:42.533082 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2103) took 1587 ms. Trace:
I20260709 12:24:42.543615 31059 rpcz_store.cc:276] 0709 12:24:40.946023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.532089 (+1586066us) service_pool.cc:224] Handling call
0709 12:24:42.532457 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.532675 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.532720 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.532773 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:42.532826 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.532896 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.533020 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.545645 31059 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:24:42.546331 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2105) took 1568 ms. Trace:
I20260709 12:24:42.546725 31059 rpcz_store.cc:276] 0709 12:24:40.978470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.544883 (+1566413us) service_pool.cc:224] Handling call
0709 12:24:42.545238 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.546037 (+   799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.546266 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:42.437942 31693 rpcz_store.cc:269] 0709 12:24:39.539268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.542173 (+  2905us) service_pool.cc:224] Handling call
0709 12:24:42.360138 (+2817965us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b540011b7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:42.360308 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441970944}
I20260709 12:24:42.532891 31067 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:42.534590 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:42.548671  1234 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:42.458098  1545 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.549351  1234 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:42.550284  1383 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.537983 31694 rpcz_store.cc:276] 0709 12:24:39.420656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.420782 (+   126us) service_pool.cc:224] Handling call
0709 12:24:39.445709 (+ 24927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.928652 (+482943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.483255 (+2554603us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54008273d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:42.483471 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062626176}
I20260709 12:24:42.553930 30984 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:42.485455  1544 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.532485  1268 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:42.559374 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2104) took 1590 ms. Trace:
I20260709 12:24:42.566283 31068 rpcz_store.cc:276] 0709 12:24:40.969260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.540893 (+1571633us) service_pool.cc:224] Handling call
0709 12:24:42.558948 (+ 18055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.559103 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.559128 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.559164 (+    36us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:42.559193 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.559241 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.559325 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.461053  1551 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 12:24:42.379269 30990 raft_consensus.cc:2410] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 12:24:42.548802  1530 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.078s	user 0.005s	sys 0.004s
I20260709 12:24:42.483296  1447 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:42.438410  1550 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:42.555065 30958 connection.cc:256] RPC call timeout handler was delayed by -1.0276376710003206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
I20260709 12:24:42.556229 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:42.488291  1511 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.543504  1418 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:42.577685 31062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 1789) took 3002 ms (client timeout 1671 ms). Trace:
I20260709 12:24:42.578173 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1645) took 2603 ms. Trace:
I20260709 12:24:42.578629 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1954) took 2563 ms. Trace:
I20260709 12:24:42.579865 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:42.604919 31059 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.531867  1546 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:42.602198 31062 rpcz_store.cc:269] 0709 12:24:39.567076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.567502 (+   426us) service_pool.cc:224] Handling call
0709 12:24:42.568874 (+3001372us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001a77d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:42.569024 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470943104}
I20260709 12:24:42.609505 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:42.603083 31143 rpcz_store.cc:276] 0709 12:24:39.964782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.964947 (+   165us) service_pool.cc:224] Handling call
0709 12:24:39.974520 (+  9573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.029625 (+1055105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.568029 (+1538404us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001ea0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:42.568263 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2424155008}
I20260709 12:24:42.604043 31141 rpcz_store.cc:276] 0709 12:24:40.009965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.054268 (+ 44303us) service_pool.cc:224] Handling call
0709 12:24:40.148054 (+ 93786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.496042 (+347988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.572944 (+2076902us) spinlock_profiling.cc:243] Waited 48.6 ms on lock 0x7b540001b150. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:42.573107 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111723776}
I20260709 12:24:42.598603 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:42.611883 31061 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:42.608565 31695 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 825) took 3189 ms (client timeout 2173 ms). Trace:
I20260709 12:24:42.555577  1470 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:42.615300 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:42.567783  1545 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:42.612920 31695 rpcz_store.cc:269] 0709 12:24:39.419605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.419752 (+   147us) service_pool.cc:224] Handling call
0709 12:24:42.556028 (+3136276us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:42.608476 (+ 52448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4827392}
I20260709 12:24:42.540405  1536 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.595809  1447 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:42.623063 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.017s
I20260709 12:24:42.625352 31062 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:42.631145 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.001s	sys 0.017s
W20260709 12:24:42.631424 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 510381us
I20260709 12:24:42.632232 31059 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:42.566968  1544 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.642450 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:42.624339 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:42.643208 31143 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.602269  1268 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.609221  1418 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.542816  1512 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.646118 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:42.647364 31068 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:42.648130 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 742) took 1769 ms. Trace:
I20260709 12:24:42.649453 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:42.650537 31059 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:42.644706 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 491913us
I20260709 12:24:42.655553 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:42.598404  1234 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:24:42.648705 31698 rpcz_store.cc:276] 0709 12:24:40.874069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.930836 (+ 56767us) service_pool.cc:224] Handling call
0709 12:24:40.931180 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.931552 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.931579 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.946144 (+ 14565us) log.cc:844] Serialized 26 byte log entry
0709 12:24:42.107676 (+1161532us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:42.624389 (+516713us) log.cc:844] Serialized 14 byte log entry
0709 12:24:42.642902 (+ 18513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.643063 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.643139 (+    76us) raft_consensus.cc:1685] finished
0709 12:24:42.643150 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.643343 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55140,"prepare.queue_time_us":403375,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":2953,"raft.queue_time_us":4530,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":8835,"thread_start_us":1449089,"threads_started":2,"wal-append.queue_time_us":1156837}
I20260709 12:24:42.665516 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:42.599292  1530 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.644656  1447 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.623857  1545 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:42.683342 31059 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:42.683801 31109 connection.cc:256] RPC call timeout handler was delayed by -1.2671901420003451s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 12:24:42.673892 31137 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
I20260709 12:24:42.654174   970 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:42.643743  1486 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:42.653972  1470 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.683475   953 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:42.649371  1268 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.712723 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:42.714020 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:42.689267  1552 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.721251 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:42.699357 31137 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:42.718552 31062 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:42.621130  1546 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.697557  1535 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.724408 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:42.698266  1246 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.642700  1405 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.737689 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:42.720690 31107 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.024s	sys 0.048s
W20260709 12:24:42.769534 31107 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.025s	sys 0.048s
I20260709 12:24:42.736414 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:42.694787  1545 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:42.743816  1521 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.520s	user 0.010s	sys 0.002s
I20260709 12:24:42.810477  1521 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:42.717633  1470 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.714753   970 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.746886  1246 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.827987  1246 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.828529  1246 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:42.743642  1535 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.828876  1246 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:42.829300  1246 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.777809  1507 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:42.830435  1535 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.830546  1507 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:42.727015  1552 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.742400  1475 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.772820  1556 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.814484 31061 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:42.725867  1573 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.768033  1234 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.823555 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:42.843869 31063 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.825471  1511 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:42.781148  1508 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.770932  1569 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:42.826076  1470 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:42.719929   953 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.831823 31137 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:42.717443  1486 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.773104  1155 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.831080  1535 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:42.831095  1560 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:24:42.813596  1566 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.814913  1559 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.886515 31063 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:42.831188  1507 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.911192 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2108) took 1585 ms. Trace:
I20260709 12:24:42.875974  1550 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.912755 31063 rpcz_store.cc:276] 0709 12:24:41.325681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.823263 (+1497582us) service_pool.cc:224] Handling call
0709 12:24:42.911116 (+ 87853us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.913775 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 480) took 4190 ms. Trace:
I20260709 12:24:42.857334  1571 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:42.915040 30989 rpcz_store.cc:276] 0709 12:24:38.718480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.739884 (+ 21404us) service_pool.cc:224] Handling call
0709 12:24:38.740162 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.341318 (+601156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.908755 (+3567437us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54000767d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:42.908942 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1380974848}
I20260709 12:24:42.918093  1571 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:42.930303  1558 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:24:42.854607  1234 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.875523  1508 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.876355  1569 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.935048  1561 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:42.935636  1561 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.938849  1268 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.939128  1545 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:42.939395 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2090) took 2346 ms. Trace:
I20260709 12:24:42.939797 30986 rpcz_store.cc:276] 0709 12:24:40.593508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.593714 (+   206us) service_pool.cc:224] Handling call
0709 12:24:40.594041 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.669264 (+ 75223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.669310 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:40.669366 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:40.669413 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.939164 (+2269751us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b5400022bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:42.939211 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.939337 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172399360}
I20260709 12:24:42.951815 31143 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:24:42.956473 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 220125us
I20260709 12:24:42.957341  1507 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:42.958169  1571 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.959357 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1806) took 2134 ms. Trace:
I20260709 12:24:42.960775 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:42.959789 31058 rpcz_store.cc:276] 0709 12:24:40.810169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.863530 (+ 53361us) service_pool.cc:224] Handling call
0709 12:24:40.878760 (+ 15230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.028140 (+149380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.028178 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:42.315704 (+1287526us) log.cc:844] Serialized 26 byte log entry
0709 12:24:42.907814 (+592110us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:42.907856 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.907941 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.944260 (+ 36319us) raft_consensus.cc:1685] finished
0709 12:24:42.944279 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.944394 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18699,"raft.queue_time_us":59292,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":17846,"thread_start_us":589425,"threads_started":2,"wal-append.queue_time_us":504527}
I20260709 12:24:42.971513 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:42.979393 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:42.918833  1567 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.980641 30990 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
W20260709 12:24:42.962517  1565 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:24:42.985359 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:42.980710 30988 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:42.992764 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:42.989960  1594 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:42.845655  1556 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.958488  1314 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:24:42.997750 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:42.914181  1535 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.998701 30992 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:24:42.957341   953 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:42.999958 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:42.997025 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:43.000837 30990 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:43.000872 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:42.917786  1559 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.003315 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.004690  1559 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:43.005247 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:43.005853 31695 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:42.845572  1573 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:42.997843 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 342179us
I20260709 12:24:43.006311 30991 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:24:42.993973 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:42.998667 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 372524us
I20260709 12:24:42.999800 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:43.008271 31062 raft_consensus.cc:2410] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:24:42.996680 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.008775 31694 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:42.984455  1512 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.012614 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:43.013342 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:43.016454 31059 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:43.016974 30988 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:43.013893 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:43.018435 30983 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:43.007277 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:43.014456 31063 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:42.980747  1536 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:42.978667  1418 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:42.990567  1567 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:42.918884  1566 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.021703 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:43.022588  1551 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 12:24:43.169335 30988 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:43.050662 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.171602 31693 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:43.170773 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.173444 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:43.046764 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:43.172199  1556 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:43.167161  1234 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:43.248626  1447 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
I20260709 12:24:43.249073  1447 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:43.176726 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:43.045719 30986 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:43.251909 31061 raft_consensus.cc:2368] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:24:43.252543  1234 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.252986  1234 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.253364  1234 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:43.096741 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 46370.6us
I20260709 12:24:43.255141 31109 leader_election.cc:400] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
I20260709 12:24:43.255404 31109 leader_election.cc:403] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:24:43.251824 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 75205.2us
I20260709 12:24:43.256520 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:43.257275  1559 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:43.257642  1559 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.257965  1559 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:43.178412 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 171231us
I20260709 12:24:43.258664 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:43.259548  1563 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5.
I20260709 12:24:43.259796 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:43.261327 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:43.261720  1585 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:43.261986  1263 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:43.085295 30990 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:43.263139 30992 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:43.264205  1234 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.265041  1566 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.097254 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.265978 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.266566 31696 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:43.266949 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.267407 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.065500 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:43.186563 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 139858us
I20260709 12:24:43.242434 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.268711 31693 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:43.243362 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.269445 31136 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:43.270208  1480 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:43.270509 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.121780 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.239778  1551 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:43.246452 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:43.246817 31723 heartbeater.cc:646] Failed to heartbeat to 127.29.37.126:44053 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.29.37.126:44053 timed out after 15.000s (SENT)
I20260709 12:24:43.025192 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:43.250253  1578 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:43.292338 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.251749 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 740) took 2403 ms. Trace:
I20260709 12:24:43.294358 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:43.294857 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.268115 31696 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:43.267984 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:43.271898 31058 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:43.296105 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:43.296607  1559 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:43.161608 31068 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.002s
I20260709 12:24:43.299008  1559 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:43.299125 31068 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:43.055691  1589 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.255s
I20260709 12:24:43.300261 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:43.300490 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:43.300599  1576 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:43.109620 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.300930 31693 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:43.301169  1576 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:43.301440 31690 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:43.103238  1591 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.204s
I20260709 12:24:43.302132  1576 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.303681  1576 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.268126 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:43.304901 31694 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.305887  1589 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:43.272918 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:43.306855 31061 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:43.309729 31693 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:43.092439 30983 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:43.314730 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:43.314790 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.314730 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:43.315919 30983 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:43.316061 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.276188 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:43.316915 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:43.273758 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:43.270675  1515 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:43.319639 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 46114.6us
I20260709 12:24:43.320533 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:43.010458  1571 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:43.037724  1594 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:43.197638 31695 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:43.187556 30988 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:43.311982 31692 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:43.156473  1512 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.294234 31689 rpcz_store.cc:276] 0709 12:24:40.847919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.882942 (+ 35023us) service_pool.cc:224] Handling call
0709 12:24:40.948210 (+ 65268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.360650 (+412440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.251500 (+1890850us) spinlock_profiling.cc:243] Waited 95.7 ms on lock 0x7b540076ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:43.251693 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220160768}
I20260709 12:24:43.312485 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:43.358857 31063 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:43.292101  1234 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:43.315955 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.323222  1578 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:43.020319  1507 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.305s
I20260709 12:24:43.112485  1569 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:43.373984 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.330462  1418 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:43.318367 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:43.376571 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58269.6us
I20260709 12:24:43.270910  1480 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.380761  1567 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.112731  1570 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.275s
I20260709 12:24:43.381359 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:43.381929  1570 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:43.382293  1570 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:24:43.383687 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 63249.9us
I20260709 12:24:43.129207   970 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:43.386394 30988 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:43.341859 30991 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:43.105793  1573 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:43.111236  1535 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.044720  1588 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:43.356707 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:43.123392  1561 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.327818  1246 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.123726  1416 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.222s
W20260709 12:24:43.358253 31692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 735) took 3458 ms (client timeout 1805 ms). Trace:
I20260709 12:24:43.121960  1590 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.260s
I20260709 12:24:43.358858  1263 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:43.323699  1364 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:43.371767  1512 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:43.110284  1546 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:43.323374  1551 raft_consensus.cc:2764] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:43.385022 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:43.385901 31060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.012s	sys 0.022s
W20260709 12:24:43.428946 31060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.013s	sys 0.023s
W20260709 12:24:43.429342 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44578.9us
I20260709 12:24:43.429849 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2101) took 2500 ms. Trace:
I20260709 12:24:43.430285 31060 rpcz_store.cc:276] 0709 12:24:40.929353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.463542 (+1534189us) service_pool.cc:224] Handling call
0709 12:24:42.519166 (+ 55624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.519349 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:42.831962 (+312613us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.429336 (+597374us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:43.429390 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:43.429439 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.429519 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.429796 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":433450,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":891198,"threads_started":2}
I20260709 12:24:43.393821 31106 leader_election.cc:400] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:24:43.432462 31106 leader_election.cc:403] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:24:43.394486 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:43.395936  1571 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:43.396101 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.397233 30990 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:43.383679  1515 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:43.395648 31694 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:43.399073 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:43.329988  1447 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.417703 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 435) took 4334 ms. Trace:
I20260709 12:24:43.433154 31139 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:43.404345  1590 consensus_queue.cc:1243] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:43.405380  1364 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.483652 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:43.489197 31063 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.393386  1585 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260709 12:24:43.492146 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 93165.5us
W20260709 12:24:43.503273 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 109615us
W20260709 12:24:43.452199 31135 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.010s	sys 0.009s
W20260709 12:24:43.504513 31135 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.011s	sys 0.009s
I20260709 12:24:43.504914 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:43.441222  1263 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:43.440979  1583 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:43.506413 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:43.513684 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 452) took 1380 ms. Trace:
W20260709 12:24:43.507102 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 77005us
I20260709 12:24:43.506953  1314 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:24:43.383129  1570 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.444387  1515 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:43.457574 31144 rpcz_store.cc:276] 0709 12:24:39.070153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.070309 (+   156us) service_pool.cc:224] Handling call
0709 12:24:39.070624 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.142566 (+ 71942us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:39.637625 (+495059us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b18003546a0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf 00007f2eeb8ae038
0709 12:24:39.906546 (+268921us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.154583 (+248037us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:40.154654 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:40.154723 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.312751 (+3158028us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b18003546a0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:24:43.403906 (+ 91155us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b5400063e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:43.403972 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.404123 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22922,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":3498,"raft.queue_time_us":262523,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"spinlock_wait_cycles":217579136,"thread_start_us":514180,"threads_started":2,"wal-append.queue_time_us":456037,"wal-append.run_cpu_time_us":2204,"wal-append.run_wall_time_us":1396564}
W20260709 12:24:43.469758  1327 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.009s	sys 0.032s
I20260709 12:24:43.444490  1557 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:43.498890  1364 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:43.525005 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:43.507503  1536 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:43.528158 30983 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:43.527026 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.539558 31694 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:43.536871 31063 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
W20260709 12:24:43.402029 31692 rpcz_store.cc:269] 0709 12:24:39.899747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.903070 (+  3323us) service_pool.cc:224] Handling call
0709 12:24:43.121595 (+3218525us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876feee 000000000036eb39 00007f2ef07e31e0 00007f2ef08ac0c4 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:43.358155 (+236560us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23924608}
I20260709 12:24:43.541014 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:43.541149  1593 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:43.520295  1327 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.009s	sys 0.032s
I20260709 12:24:43.550029 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.518595  1569 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.520259  1515 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.560365 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.573076 31696 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:43.563791 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.596062 31144 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:43.597697 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.560317  1509 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:43.559953  1593 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:43.529659  1585 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:24:43.543962  1263 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:24:43.610502 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:43.565034  1491 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:43.594679  1536 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:43.591977  1589 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:43.419796  1594 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.617106 31063 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:43.628618 31144 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:43.603785 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.400533  1588 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:43.618631 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 344049us
I20260709 12:24:43.625658 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:43.664144 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:43.667755 30983 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:43.503640  1578 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.520097 31135 rpcz_store.cc:276] 0709 12:24:42.133568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.336037 (+202469us) service_pool.cc:224] Handling call
0709 12:24:42.336402 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.336642 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:42.880163 (+543521us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.505150 (+624987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:43.505218 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:43.505296 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.505692 (+   396us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.513610 (+  7918us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124886,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":26520,"raft.queue_time_us":7489,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5193,"thread_start_us":1098129,"threads_started":2}
W20260709 12:24:43.709661 31033 connection.cc:256] RPC call timeout handler was delayed by -1.8196374409999407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2437s for a response
I20260709 12:24:43.710853 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.611294  1593 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.712385 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:43.714864 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.715786 31135 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:43.618100  1509 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:43.717378 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:43.718176 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:43.710157 31033 connection.cc:256] RPC call timeout handler was delayed by -1.7584374409998418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2505s for a response
W20260709 12:24:43.715509  1595 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 12:24:43.631779 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:43.721747 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.764911 31135 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:43.719995 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:43.749737  1551 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:43.619033  1447 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:43.710803  1585 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:43.715368 31689 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
W20260709 12:24:43.783142 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 63233.6us
I20260709 12:24:43.783907 31694 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:43.783900 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:43.786504 31135 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:24:43.721527  1596 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:24:43.719444  1480 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.742372  1405 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:43.788589 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:43.780117 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 477271us
I20260709 12:24:43.810690 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:43.811920   981 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:43.812263  1263 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:43.814635 31063 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:43.817349 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:43.817883 31062 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:43.818794 31060 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:43.821107 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:43.820559 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:43.822443 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:43.783192  1512 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.822897 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:43.828620 31135 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:43.836374 31139 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:43.837592 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:43.785753  1447 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:43.828941 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:43.837821 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:43.616394  1491 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:43.760955  1246 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:43.807418  1405 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.841437 31139 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:43.787294  1525 raft_consensus.cc:2764] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:43.843200  1405 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:43.853715  1446 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.854713  1383 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.862310  1246 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:43.840662  1507 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:43.882184  1246 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.882645  1246 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:43.879822  1446 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:43.883448  1446 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.886675  1606 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:43.823370  1263 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:24:43.820765   981 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:43.891104   981 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.891633  1544 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:43.893592  1353 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:43.820887  1480 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.839478  1576 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:24:43.785912  1585 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.840538  1535 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.823542  1561 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:43.740895  1589 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.871480 31063 pending_rounds.cc:85] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:24:43.906646 31063 pending_rounds.cc:107] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:24:43.906935 31063 raft_consensus.cc:2889] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.871654 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1661) took 1317 ms. Trace:
I20260709 12:24:43.726642  1509 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:43.907656 31141 rpcz_store.cc:276] 0709 12:24:42.554150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.686429 (+132279us) service_pool.cc:224] Handling call
0709 12:24:42.697614 (+ 11185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:42.737768 (+ 40154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.737799 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:42.772735 (+ 34936us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.018696 (+245961us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:43.037043 (+ 18347us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.442289 (+405246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.442372 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.871446 (+429074us) raft_consensus.cc:1685] finished
0709 12:24:43.871466 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.871588 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89082,"prepare.queue_time_us":26160,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":253190,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":168,"thread_start_us":520573,"threads_started":2,"wal-append.queue_time_us":703943}
I20260709 12:24:43.785063  1551 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:43.882586  1507 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:43.900930  1585 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:43.895813  1353 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:43.901527  1405 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:43.983888 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1952) took 4086 ms. Trace:
I20260709 12:24:43.905171  1515 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.984264 31140 rpcz_store.cc:276] 0709 12:24:39.897604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.897777 (+   173us) service_pool.cc:224] Handling call
0709 12:24:39.929129 (+ 31352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.004206 (+ 75077us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:40.096271 (+ 92065us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.550088 (+453817us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:40.550157 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.550239 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.983672 (+3433433us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b54000947d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:43.983716 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.983833 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2243,"prepare.queue_time_us":98788,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":14684,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":14407,"spinlock_wait_cycles":171972992,"thread_start_us":390463,"threads_started":1,"wal-append.queue_time_us":510509,"wal-append.run_cpu_time_us":1750,"wal-append.run_wall_time_us":1145731}
I20260709 12:24:43.987214 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1662) took 1432 ms. Trace:
I20260709 12:24:43.987651 31137 rpcz_store.cc:276] 0709 12:24:42.554500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.734935 (+180435us) service_pool.cc:224] Handling call
0709 12:24:42.735402 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.882090 (+146688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.986967 (+1104877us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b54000942d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:43.987154 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220887936}
I20260709 12:24:44.002557  1548 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.901829  1535 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:43.901975  1480 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:43.881048  1602 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:43.944671 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 500) took 1801 ms (client timeout 2117 ms). Trace:
I20260709 12:24:43.978736  1353 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:43.982578  1510 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 3.344s	user 0.015s	sys 0.008s
I20260709 12:24:44.005685 31143 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:43.973418  1597 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 12:24:43.945166  1572 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:44.012087 30984 rpcz_store.cc:269] 0709 12:24:42.143289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.143388 (+    99us) service_pool.cc:224] Handling call
0709 12:24:43.934790 (+1791402us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540020f154. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:43.944603 (+  9813us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941541248}
I20260709 12:24:43.982472  1246 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.011389  1535 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.013172  1602 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:44.003235  1548 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.019598  1572 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:43.916741  1536 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:44.073571  1607 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:24:44.069203  1364 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.001s
I20260709 12:24:44.089884 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1792) took 4513 ms. Trace:
I20260709 12:24:44.074061  1602 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:44.086501  1364 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.094588  1364 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.101087 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:44.102041 31696 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:24:44.112279  1608 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:24:44.083913  1353 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.114248  1353 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:44.090284 31064 rpcz_store.cc:276] 0709 12:24:39.572507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.812390 (+239883us) service_pool.cc:224] Handling call
0709 12:24:39.841032 (+ 28642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.816744 (+975712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.085630 (+3268886us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b5400385254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:44.085800 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192131456}
I20260709 12:24:44.080991  1588 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:41.887606 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:44.124161 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 286373us
I20260709 12:24:44.015164  1491 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.125548 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:44.127110 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 289558us
I20260709 12:24:44.128695  1521 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:44.129490 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:44.129238  1521 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.132620 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.133203 31064 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:44.135004 31058 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:24:44.137253 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.137462 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:44.137840 31065 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:44.140164 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:44.128159  1486 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:44.126906  1463 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.148113 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:44.148510 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:44.151971 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:44.127336  1588 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:44.154763 30986 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:44.153724 30988 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:44.155025 30983 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:44.083631  1507 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.159859 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:44.134212  1502 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.160359 30992 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:44.154804 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:44.155082 30990 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.168817 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:44.159838  1434 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:44.168757  1598 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.227s	user 0.000s	sys 0.005s
I20260709 12:24:44.177328  1246 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.145182  1164 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:24:44.170007  1241 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:44.179391  1241 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.165314  1527 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:44.181246 30990 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:44.145539  1412 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:44.160831  1525 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:44.187220  1525 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.187682  1525 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:44.200547  1551 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:44.203032 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:44.203353  1299 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:44.203828  1299 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:44.161376  1263 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:24:44.204612  1299 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.205884  1299 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:44.207357 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52661.1us
I20260709 12:24:44.209043 30983 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:44.206697  1241 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:44.206583 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1921) took 4098 ms (client timeout 2505 ms). Trace:
I20260709 12:24:44.226702 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:44.228050 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.222409 30987 rpcz_store.cc:269] 0709 12:24:40.107696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.108470 (+   774us) service_pool.cc:224] Handling call
0709 12:24:44.206361 (+4097891us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b5400226d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:44.206497 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5455796480}
I20260709 12:24:44.141841  1446 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.211201  1412 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.153932  1418 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:24:44.200547  1480 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:44.206513  1620 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:44.219689  1434 raft_consensus.cc:2764] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:44.206073  1617 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:44.173111  1588 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.229717  1463 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.221802  1594 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:24:44.238112  1412 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.293169 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:44.291365 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 140092us
I20260709 12:24:44.303467 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.306794 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:44.307725 31059 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:44.310973 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:44.311676 31060 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:24:44.323195 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.324195 31061 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:44.229555  1615 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:24:44.328291 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.286444  1561 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:44.328878 31068 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:44.181401  1527 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.152994  1572 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.294328 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:44.292193  1548 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:44.326924  1585 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.327927  1263 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:44.336084 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:44.292757  1463 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.349659 31061 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:44.350656  1572 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:44.386477 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 182231us
I20260709 12:24:44.350431  1263 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.354031 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:44.354404  1614 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:24:44.355207 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:44.356346 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.389887 31059 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:44.364936 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.366489 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.348791 31691 raft_consensus.cc:2376] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9. Re-sending same reply.
I20260709 12:24:44.328423  1246 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:44.326663  1616 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.374576 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 509) took 1366 ms. Trace:
I20260709 12:24:44.394960 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.376863 31060 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:44.380530  1521 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.382133  1588 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.385474 31137 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:44.392091 31697 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 814) took 5325 ms (client timeout 1635 ms). Trace:
I20260709 12:24:44.391022 31062 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:44.399628 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.395246 30989 rpcz_store.cc:276] 0709 12:24:43.008483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.243054 (+234571us) service_pool.cc:224] Handling call
0709 12:24:43.243377 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.373291 (+129914us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:43.612277 (+238986us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.009197 (+396920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.009246 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:44.009298 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.374357 (+365059us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:44.374403 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.374520 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":435768,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":39,"raft.queue_time_us":91,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":11876,"spinlock_wait_cycles":298217856,"thread_start_us":633309,"threads_started":2,"wal-append.queue_time_us":529821}
I20260709 12:24:44.416814 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:44.415531 31697 rpcz_store.cc:269] 0709 12:24:39.062278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.903029 (+840751us) service_pool.cc:224] Handling call
0709 12:24:44.385286 (+4482257us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b5400738cd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:44.387470 (+  2184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790753664}
I20260709 12:24:44.387168  1585 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.420301  1585 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:44.393633  1569 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.800s	user 0.007s	sys 0.003s
I20260709 12:24:44.436105 31061 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
W20260709 12:24:44.414717 31691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 741) took 3557 ms (client timeout 1597 ms). Trace:
W20260709 12:24:44.437520 31691 rpcz_store.cc:269] 0709 12:24:40.848315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.924067 (+ 75752us) service_pool.cc:224] Handling call
0709 12:24:44.261763 (+3337696us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:44.405208 (+143445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14760192}
I20260709 12:24:44.423444 31063 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:44.440739  1446 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:44.387053  1509 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:44.417032 31060 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:44.448115 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:44.448714 31061 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.415123  1246 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:44.461583  1593 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.000s
I20260709 12:24:44.463042  1246 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:44.462720  1593 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.463794 31064 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.473717 31137 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:44.415324  1616 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.478281  1509 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.479089  1546 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:44.479476  1546 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:44.480319  1546 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.488229  1546 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.385862  1535 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.491894  1552 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.832952  1246 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.836942 31060 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:44.813690 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:44.770407 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.835815 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:44.501430 31136 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:44.622716 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:44.852289 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 229643us
I20260709 12:24:44.852311 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:44.854173 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:44.642504 31061 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:44.861912 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:44.864464 31062 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:44.864336 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:44.865123 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:44.867830 31063 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:44.836613  1434 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:44.869110 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:44.551700 31064 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:44.868569 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:44.813941  1551 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:44.677592  1263 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.871285 30983 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:44.519006  1515 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.873062 30987 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:24:44.870585  1507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.012s	sys 0.021s
I20260709 12:24:44.873665 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2095) took 4258 ms. Trace:
W20260709 12:24:44.902880 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 268941us
I20260709 12:24:44.903615 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:44.874090 31664 connection.cc:256] RPC call timeout handler was delayed by -2.0107960479999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
I20260709 12:24:44.875618 31138 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:24:44.904726 31664 connection.cc:256] RPC call timeout handler was delayed by -1.9288960479998423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
W20260709 12:24:44.905037 31664 connection.cc:256] RPC call timeout handler was delayed by -1.9288960479998423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
W20260709 12:24:44.905299 31664 connection.cc:256] RPC call timeout handler was delayed by -1.9126960479998161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
W20260709 12:24:44.905570 31664 connection.cc:256] RPC call timeout handler was delayed by -1.7137960479999492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
I20260709 12:24:44.905524 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:44.494333  1241 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:44.900998 30985 rpcz_store.cc:276] 0709 12:24:40.614807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.614937 (+   130us) service_pool.cc:224] Handling call
0709 12:24:40.615211 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.650715 (+ 35504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.650747 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.651208 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.058429 (+407221us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:41.058465 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.839310 (+3780845us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b54002c1d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:44.839370 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.839427 (+    57us) raft_consensus.cc:1685] finished
0709 12:24:44.839445 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.873565 (+ 34120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9187,"raft.queue_time_us":10163,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":59489536,"thread_start_us":406878,"threads_started":1,"wal-append.queue_time_us":1524780,"wal-append.run_cpu_time_us":1989,"wal-append.run_wall_time_us":1112459}
I20260709 12:24:44.627733  1620 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:44.517590  1566 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.862741  1606 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.888667  1359 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:44.835610  1509 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:44.503458  1593 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:44.871376 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.890699 31143 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:24:44.874171  1327 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:24:44.810884  1586 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.443182  1569 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:44.891961 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:44.893010 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:44.587682  1563 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:44.872588 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:44.895545 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:44.888890  1299 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.003s
I20260709 12:24:44.811792  1583 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:44.873356 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:44.906735  1263 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:24:44.622833  1487 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:44.898870 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:44.883126  1412 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.000s
I20260709 12:24:44.869912  1552 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:44.905997 31664 connection.cc:256] RPC call timeout handler was delayed by -1.6687960479998765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
W20260709 12:24:45.050781 31664 connection.cc:256] RPC call timeout handler was delayed by -1.6615960479998648s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
W20260709 12:24:45.051123 31664 connection.cc:256] RPC call timeout handler was delayed by -1.6588960479998605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
W20260709 12:24:45.051385 31664 connection.cc:256] RPC call timeout handler was delayed by -1.5355960480001158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
W20260709 12:24:45.051658 31664 connection.cc:256] RPC call timeout handler was delayed by -1.4932960480000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
W20260709 12:24:45.052012 31664 connection.cc:256] RPC call timeout handler was delayed by -1.4932960480000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
I20260709 12:24:45.053664 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:45.054979 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:44.506318  1508 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.627699  1493 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.267s
I20260709 12:24:44.518052  1615 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:45.056641  1570 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.056861  1615 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:44.928148  1606 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.058254  1412 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.059260  1615 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:45.059804  1615 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:45.059980  1606 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:45.060413  1412 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:44.909318 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:44.873309  1556 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:44.876636  1434 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:44.911823 31067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 635) took 5320 ms (client timeout 1589 ms). Trace:
I20260709 12:24:44.898862  1633 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:44.919479 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:44.495425  1525 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:44.839064  1246 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:44.906543  1507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.012s	sys 0.021s
I20260709 12:24:44.907078  1241 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:45.009953 31061 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:44.910856  1620 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:45.013360  1359 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:45.013885 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.045421  1640 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.082348 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.083288 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:45.083801 31144 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.898121 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:45.014783 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 160694us
I20260709 12:24:45.086141 31139 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:45.086490 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.016674 30991 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:45.086978 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:44.897078  1551 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.088163 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.089088 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.089542  1551 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.524377  1527 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:44.936612  1530 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.017757  1588 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:45.096556 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:45.097661 31061 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:45.015717  1563 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.014331  1463 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:45.066092 31067 rpcz_store.cc:269] 0709 12:24:39.586570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.587658 (+  1088us) service_pool.cc:224] Handling call
0709 12:24:44.906722 (+5319064us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b54002a2954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:44.906842 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6752541440}
I20260709 12:24:45.060528  1606 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.926322  1566 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:45.061297  1602 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.938s	user 0.005s	sys 0.005s
W20260709 12:24:45.063321  1546 consensus_meta.cc:287] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.004s
I20260709 12:24:45.010588  1616 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.006246  1586 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.060290  1615 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:45.080809 31135 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:45.089968 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:45.122592 31139 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:45.093554 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:45.095058 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.064404 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 196892us
I20260709 12:24:45.124750 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:45.045266  1531 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:24:45.099038 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.076032  1552 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:45.100977  1359 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:45.104341 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.019281  1583 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.107379 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.085218  1633 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:45.118945 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.069039  1556 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:45.121482 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 210104us
I20260709 12:24:45.100281  1493 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:45.132131 31063 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:45.132366 31139 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:45.137384 31067 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
W20260709 12:24:45.135113 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 288713us
W20260709 12:24:45.139816 31109 leader_election.cc:336] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.635s (SENT)
I20260709 12:24:45.140172 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:45.140611 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.141944 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.142565 31063 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:45.143435 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.143519 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:45.129958 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.120908  1641 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:45.114683  1434 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.185604 31062 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:45.186604 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.187088 31062 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:45.149870 30955 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:45.147444  1359 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.134408  1546 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:45.151741  1615 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.152477  1559 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:45.194701 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.153421 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1972) took 1005 ms. Trace:
I20260709 12:24:45.195865 31137 rpcz_store.cc:276] 0709 12:24:44.147768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.147899 (+   131us) service_pool.cc:224] Handling call
0709 12:24:45.153266 (+1005367us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54001e8cd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:45.153368 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489041664}
I20260709 12:24:45.153999  1586 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:45.137907  1642 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:45.247287 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:45.247828 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.154731 31143 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:45.155722 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.248893 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:45.249320 31058 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:45.156575  1470 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:24:45.157505 31109 leader_election.cc:336] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.170s (SENT)
I20260709 12:24:45.251240 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.158943 30991 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:45.158576  1552 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.160251  1525 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.160672 30985 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:45.160808 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:45.162352  1638 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:45.163044  1636 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:24:45.163810  1446 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.164584  1563 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:45.134311  1633 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:45.148272  1493 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:45.167923  1591 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:45.127101  1241 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.170318 31138 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 2.
I20260709 12:24:45.118444  1602 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.171365  1556 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:45.122624  1535 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:45.103439  1588 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.129324  1616 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.161952 30986 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:24:45.176548  1641 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:45.427649  1551 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:45.427865  1641 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.428098  1551 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:45.430508 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.430462  1493 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.431823 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:45.432550  1602 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:45.433096  1241 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.383262  1591 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:45.383733  1525 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.428937  1551 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.177433  1246 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:45.178076  1620 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.434854  1525 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.179281  1606 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:45.435926  1606 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.436945 31693 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:45.311568  1530 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.151744  1566 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.191871 30955 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.253726  1411 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.162s
W20260709 12:24:45.192925 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 136943us
I20260709 12:24:45.203485 31144 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:45.455555 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.215296 31139 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:24:45.160269  1521 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:24:45.251606 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:45.197566  1463 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:45.256302  1470 consensus_queue.cc:1243] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:45.256487 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.256826  1508 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:45.250402  1572 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.861s	user 0.011s	sys 0.000s
I20260709 12:24:45.193815  1546 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:45.318573 31058 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:45.251616  1434 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.385099 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 1818) took 1030 ms. Trace:
I20260709 12:24:45.491322 31068 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:45.491098 31064 rpcz_store.cc:276] 0709 12:24:44.354777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.354946 (+   169us) service_pool.cc:224] Handling call
0709 12:24:45.384870 (+1029924us) spinlock_profiling.cc:243] Waited 74.3 ms on lock 0x7b5400384354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:45.385014 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170834176}
W20260709 12:24:45.399578 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 90398.3us
I20260709 12:24:45.492396 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.175277 31661 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:45.415709 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:45.657383 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 482176us
W20260709 12:24:45.659191 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.660274  1434 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:45.660681  1434 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:45.661070  1434 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.260649 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.420281 31065 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:24:45.663506 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 664) took 1072 ms. Trace:
I20260709 12:24:45.663735 31691 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:45.663815 31065 rpcz_store.cc:276] 0709 12:24:44.591175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.167096 (+575921us) service_pool.cc:224] Handling call
0709 12:24:45.419847 (+252751us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b5800129398. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:45.420056 (+   209us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.663282 (+243226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.663456 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47969536}
I20260709 12:24:45.665009  1551 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.665194 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:45.425822 31689 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:45.430073 31140 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:45.315404  1643 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:45.449196 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 257375us
I20260709 12:24:45.152664 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:45.450766  1646 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:45.422134  1585 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:24:45.464150  1463 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:45.458369  1593 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.438601  1530 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.331902  1446 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.466789  1411 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:24:45.467455 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 309952us
I20260709 12:24:45.468021  1527 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:45.475940 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.483520 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1678) took 2235 ms. Trace:
I20260709 12:24:45.679853 31136 rpcz_store.cc:276] 0709 12:24:43.248462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.514196 (+265734us) service_pool.cc:224] Handling call
0709 12:24:43.542836 (+ 28640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.840885 (+1298049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.483208 (+642323us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b540006fed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:45.483450 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2204053248}
I20260709 12:24:45.458988  1650 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.460513  1493 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.683770 31136 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:45.683513  1463 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.685268  1470 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.490381 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:45.685720  1470 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.493610  1525 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.494004 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.498456 31141 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.687319 31690 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:24:45.687417 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:45.687700  1470 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:45.688503  1470 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.688725 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:45.507174 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:45.689858 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 182794us
I20260709 12:24:45.690541 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:45.462967  1583 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.470999  1588 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:45.485396  1508 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:45.491322  1546 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:45.540434 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.542176 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.542433  1405 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:45.517351  1246 raft_consensus.cc:2764] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:45.536984  1562 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:45.536494  1307 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:45.552191 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 92982.4us
I20260709 12:24:45.557099  1641 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.565451  1640 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:45.566843 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 416474us
I20260709 12:24:45.568236  1491 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.559448  1606 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.612425 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.615664 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 1966) took 1987 ms. Trace:
I20260709 12:24:45.697510  1572 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:45.618737 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:45.619056 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.618917  1447 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.633404 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 663) took 1115 ms. Trace:
I20260709 12:24:45.713220 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:45.714915 31060 rpcz_store.cc:276] 0709 12:24:44.518489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.568574 (+ 50085us) service_pool.cc:224] Handling call
0709 12:24:44.814088 (+245514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.837348 (+ 23260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.633213 (+795865us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400384850. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:45.633347 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563670272}
I20260709 12:24:45.715735 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:45.716033 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:45.652921 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.655311 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.654606 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:45.668226 31140 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:45.669374 31065 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:24:45.669796 30955 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:45.720877 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 51230.4us
I20260709 12:24:45.721345 31065 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:45.721517 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:45.722376 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.722824 31065 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:45.723167 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:45.672796 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:45.724092 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 51414.8us
I20260709 12:24:45.724745  1353 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:45.725015 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:45.667707  1643 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.725693  1353 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:45.720124 31060 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:45.728807 31060 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:45.693429 31695 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:45.678334  1411 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:45.674055  1646 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:45.699213  1535 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:24:45.698305  1508 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.690436  1327 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:45.700207 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.699461 31141 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:45.705653 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.691241  1650 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.697830  1635 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.752619 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:45.753595   953 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:45.708667 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:45.754431   953 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:45.754524  1472 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:45.754848   953 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.755299   953 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.710091 31135 rpcz_store.cc:276] 0709 12:24:43.628635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.840775 (+212140us) service_pool.cc:224] Handling call
0709 12:24:43.841063 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.122077 (+1281014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.615414 (+493337us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540006f9d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:45.615595 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2850620672}
I20260709 12:24:45.712671 31693 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:45.616122 31694 raft_consensus.cc:2410] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:45.734378 31060 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:45.738886 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:45.695996 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:45.705229  1470 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:24:45.702140  1583 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:24:45.727438  1353 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.728854 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:45.755090  1472 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:45.755432  1463 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:45.769309 30987 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:45.772239  1463 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:45.772470 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.772684  1472 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:45.773313 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44526.6us
I20260709 12:24:45.773460  1511 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:45.773788 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.774055  1472 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.782666  1569 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.783365  1569 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:45.784125  1569 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:45.784430 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:45.784437  1569 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:45.785305 30989 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:45.785627 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:45.785533  1569 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:45.774433  1566 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:45.787078  1566 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.787170  1569 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:45.787533  1566 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.787943 30983 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:45.774477  1198 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.773955 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:45.789870  1635 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.787326 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:45.797022 30991 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:45.791424 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 85848.2us
I20260709 12:24:45.744191  1263 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:45.792743 31697 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.793514 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.716905  1404 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:24:45.744858  1646 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.718770  1561 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:45.724768  1530 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:45.772897  1463 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:45.735831  1640 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:45.742573  1447 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.795152 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:45.708719  1307 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:45.754568 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 46006.6us
I20260709 12:24:45.799415 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:45.727780  1358 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.744536  1537 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:45.901120  1411 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.940174  1537 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.940613  1537 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:45.940815  1411 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.808439 31058 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:45.728127  1617 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:24:45.774135  1511 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:45.747498  1406 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.790865  1531 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.955730 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:45.794055  1567 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.790752  1474 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:24:45.742157  1657 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:45.931813  1446 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.966982 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:45.795804  1606 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:45.794415  1641 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.829540  1561 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.791213  1527 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:24:45.911378 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.978786 31693 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:45.707634  1562 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:45.918900 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 107906us
W20260709 12:24:45.919070 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 176120us
I20260709 12:24:45.832855  1659 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 12:24:45.868458  1594 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:24:45.870011  1640 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:45.970741 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.819447  1234 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:45.803514  1263 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:45.926358 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:46.005632 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.927615 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:45.929276 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:46.007182 31144 raft_consensus.cc:2376] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6. Re-sending same reply.
I20260709 12:24:46.007407 31698 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:45.929415 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.980749  1527 consensus_queue.cc:1243] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:24:45.934044 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 134696us
I20260709 12:24:45.886951  1327 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
W20260709 12:24:45.923240 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 127755us
I20260709 12:24:45.937377 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.904840  1535 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.947317 31697 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:45.908416  1643 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.918296  1359 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.948597 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.954849 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:45.969302  1531 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:45.960588 31058 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.970695 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:45.954615  1511 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:46.009490  1263 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:45.984737 31109 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.147s (SENT) [suppressed 22 similar messages]
I20260709 12:24:45.982777 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:45.976321  1666 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:24:45.974148  1606 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.924682  1651 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:24:45.966538  1657 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:24:45.932660 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.012055 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:46.016122  1234 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:45.957347 30958 connection.cc:256] RPC call timeout handler was delayed by -1.1493737660002807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
W20260709 12:24:46.065254 30958 connection.cc:256] RPC call timeout handler was delayed by -1.139212506000149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.167s for a response
I20260709 12:24:46.011981 31692 raft_consensus.cc:2376] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9. Re-sending same reply.
I20260709 12:24:46.066190 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:46.036949  1650 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:45.959729  1668 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:46.047860 31139 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
W20260709 12:24:46.052737 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68039.6us
I20260709 12:24:46.056233 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:45.934208  1652 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:24:45.978965  1674 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:46.086189 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 103480us
I20260709 12:24:46.086483  1674 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:46.087421  1327 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:46.087492 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:46.088011  1327 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:46.089001 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:46.054100  1198 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.089581 31065 raft_consensus.cc:2376] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6. Re-sending same reply.
W20260709 12:24:46.066093  1670 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:24:46.061359  1606 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:46.022428  1535 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.104619 31058 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:46.063817  1657 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:46.154423 30991 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:46.025293  1359 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.105613  1650 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:46.126346 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2134) took 1601 ms. Trace:
I20260709 12:24:46.178503 31063 rpcz_store.cc:276] 0709 12:24:44.525161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.184113 (+658952us) service_pool.cc:224] Handling call
0709 12:24:46.125698 (+941585us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b5800129398. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:46.125874 (+   176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.126039 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.126064 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.126102 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:46.126131 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.126180 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.126295 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18833792}
I20260709 12:24:46.126533 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.069324   981 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.156117 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:24:46.171693 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.123428 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.194504 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:46.196215 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 181476us
W20260709 12:24:46.197768 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 185777us
I20260709 12:24:46.198889 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:46.091578  1446 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:24:46.205588 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:46.200412 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 112982us
I20260709 12:24:46.206284 31063 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:46.172542  1675 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:46.099565  1583 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:24:46.211555 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:46.212227 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.213008 31664 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:46.154179  1307 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:46.108112  1198 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.232880 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:46.235796 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:24:46.221076 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.088914  1327 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.206130  1474 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:46.092599  1668 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.105356  1640 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.228276  1650 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:46.202387   981 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.241427  1521 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:46.176761  1234 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.246805  1487 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:46.244462  1683 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:46.210330  1658 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:24:46.236464  1684 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:46.261941  1651 raft_consensus.cc:2764] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:46.260529  1474 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:46.279592 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:46.274986  1680 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 12:24:46.269280  1562 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:46.296914 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1690) took 1398 ms. Trace:
I20260709 12:24:46.297433 31138 rpcz_store.cc:276] 0709 12:24:44.898077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.905316 (+  7239us) service_pool.cc:224] Handling call
0709 12:24:46.296693 (+1391377us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54001ea5d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:46.296838 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607243520}
W20260709 12:24:46.288174 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.299163 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.294883 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:46.307586 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2138) took 1459 ms. Trace:
I20260709 12:24:46.308111 31059 rpcz_store.cc:276] 0709 12:24:44.848245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.169590 (+321345us) service_pool.cc:224] Handling call
0709 12:24:46.306714 (+1137124us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b5800129398. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:46.306938 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.307174 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.307214 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.307277 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:46.307334 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.307401 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.307527 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42580736}
I20260709 12:24:46.274542  1683 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.278554  1684 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:46.316723 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.320945 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.278661  1694 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:46.326956 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 131486us
I20260709 12:24:46.328750 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:46.327061 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 119809us
I20260709 12:24:46.332369 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:46.335381 31138 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:46.363534  1700 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:46.364198  1700 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:46.365264  1700 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.367628 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:46.367950 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:46.282359  1508 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.296142  1686 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.373978 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:46.376137 31139 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
W20260709 12:24:46.366745  1472 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.002s
I20260709 12:24:46.329727  1683 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:46.331012  1684 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:46.381654 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:46.382776 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 87381us
I20260709 12:24:46.368444  1651 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:46.377611  1493 consensus_meta.cc:287] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.694s	user 0.007s	sys 0.003s
I20260709 12:24:46.387712 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:46.366425  1694 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:46.387436 31136 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:46.373272  1446 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:24:46.390884  1535 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:46.388824  1685 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:46.394032  1521 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.400030  1487 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:46.413719  1702 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.414108  1704 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.414316  1702 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.414691  1704 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.416945  1704 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:46.417655  1702 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:46.418632  1705 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.419188  1705 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.421312  1705 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:46.402128  1359 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:46.401908 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:46.392632  1474 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.437839  1198 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:46.380321  1686 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.422181  1521 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.456238   981 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:46.592002 30958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.005s	sys 0.009s
W20260709 12:24:46.592449 30958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.005s	sys 0.009s
W20260709 12:24:46.594617 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 528459us
I20260709 12:24:46.450070  1701 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:46.598521  1692 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:24:46.597764 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:46.455317  1694 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:46.527258  1689 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:24:46.620042 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:24:46.506353  1696 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:24:46.622923 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:46.626843 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:46.628183 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:46.599128  1682 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 12:24:46.510804  1698 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:46.617444  1491 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.906s	user 0.010s	sys 0.003s
I20260709 12:24:46.630872 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:46.621119  1359 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:46.600386  1434 consensus_meta.cc:287] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.936s	user 0.013s	sys 0.000s
W20260709 12:24:46.676038 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45238.3us
W20260709 12:24:46.526942  1679 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
W20260709 12:24:46.494824  1569 consensus_meta.cc:287] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.000s
W20260709 12:24:46.647415   953 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.892s	user 0.012s	sys 0.001s
I20260709 12:24:46.682474 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.646257  1701 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:46.677819  1359 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:46.684248 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.625455  1508 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:46.654960  1699 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:46.688318 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.728224 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.725893  1650 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:46.687161 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.691442 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.691880  1359 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:46.694015  1709 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:46.748951 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:46.751303 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:46.751958 30989 raft_consensus.cc:2410] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:24:46.686129  1434 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:46.725374  1411 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.784s	user 0.005s	sys 0.005s
I20260709 12:24:46.728654 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.728914 30991 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
W20260709 12:24:46.678702  1551 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.012s	user 0.013s	sys 0.000s
W20260709 12:24:46.741382 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.748131 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:46.727048  1546 consensus_meta.cc:287] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.023s	user 0.011s	sys 0.002s
I20260709 12:24:46.755255 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:46.749893  1463 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.869s	user 0.008s	sys 0.002s
I20260709 12:24:46.760900 30987 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:46.730182  1591 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.758710  1701 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.768059  1687 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:46.760036 30989 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:46.752641  1535 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.688599   953 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.770901 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.754951  1699 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:46.801390  1712 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:46.808877  1712 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:46.785085 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:46.811640 31139 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:46.818336 30985 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:46.819460 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:46.820567 30985 raft_consensus.cc:1217] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[8.5-8.5]   Dedup: 8.5->[]
I20260709 12:24:46.820986 30985 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:46.787391 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:46.766566  1546 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.789489  1434 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:46.685837  1698 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.759191  1709 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:46.799016  1521 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:46.800974  1447 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.935s	user 0.014s	sys 0.001s
I20260709 12:24:46.836870 31061 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:24:46.837257 31061 pending_rounds.cc:85] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:24:46.837603 31061 pending_rounds.cc:107] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:24:46.837932 31061 raft_consensus.cc:2889] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:46.826555 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:46.519402  1695 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.387s
I20260709 12:24:46.835770 30983 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:46.830644  1686 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:46.831936 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:46.805092  1701 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:46.829538  1668 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:46.769503  1463 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:46.823725  1566 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.036s	user 0.007s	sys 0.006s
I20260709 12:24:46.833994 31140 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:46.839071 31144 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:24:46.798322 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.829989  1591 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:46.829097  1705 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:46.836025  1511 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.000s
W20260709 12:24:46.824082  1606 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.650s	user 0.008s	sys 0.002s
W20260709 12:24:46.843742 31032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.019s	sys 0.024s
I20260709 12:24:46.833711  1709 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:46.809975  1535 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.844187  1642 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.836553  1712 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:46.856267 31032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.019s	sys 0.024s
W20260709 12:24:46.870936 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.863897 30985 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:46.853574 30958 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:46.874759 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.866743 31068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 1817) took 2513 ms (client timeout 1566 ms). Trace:
I20260709 12:24:46.875573 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:46.875224 31068 rpcz_store.cc:269] 0709 12:24:44.353678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.353793 (+   115us) service_pool.cc:224] Handling call
0709 12:24:46.866490 (+2512697us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001bb054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:46.866662 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2536201472}
W20260709 12:24:46.892720 31694 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 470) took 1441 ms (client timeout 1571 ms). Trace:
W20260709 12:24:46.893272 31694 rpcz_store.cc:269] 0709 12:24:45.421938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.431584 (+  9646us) service_pool.cc:224] Handling call
0709 12:24:46.863301 (+1431717us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400738cd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:46.863461 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421977472}
I20260709 12:24:46.893920 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:46.850893  1648 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.862138  1263 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.801s	user 0.004s	sys 0.005s
W20260709 12:24:46.894994 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.852717  1546 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:46.809127  1699 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:46.884531  1710 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:24:46.899118 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:46.897854  1569 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:46.896576  1263 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.900718 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:46.851537  1566 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.856305  1606 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:46.906872  1711 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:24:46.911325  1263 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:46.911774 31065 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:46.905493 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:46.895803  1521 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:46.852643  1511 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.915688   953 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:46.913280  1569 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:46.917727  1463 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:46.897132  1698 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:46.935622  1511 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:46.936321  1561 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.948s	user 0.005s	sys 0.006s
I20260709 12:24:46.936384 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:46.917824  1521 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:46.951341 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.951805 31136 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:46.918738  1657 consensus_meta.cc:287] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.724s	user 0.001s	sys 0.009s
I20260709 12:24:46.937110 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:46.931496  1569 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:46.950402  1561 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.955528  1406 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:46.933009  1705 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:46.969717  1646 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.140s	user 0.010s	sys 0.001s
I20260709 12:24:46.953370  1521 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:46.966073  1537 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.022s	user 0.012s	sys 0.000s
I20260709 12:24:46.979887 31063 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:46.989082  1561 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:46.961530  1406 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:46.963045  1705 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.900821  1642 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.979485  1537 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.970161  1646 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:46.999794  1646 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:46.961545  1657 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:46.993069  1406 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:47.002468  1406 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:46.993544  1705 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:46.916121  1648 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.019271  1687 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:47.020774  1606 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:47.022562 30986 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:47.035933 30984 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:47.047389 31664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.022s	sys 0.024s
W20260709 12:24:47.014456  1675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.019s	sys 0.025s
W20260709 12:24:47.048471 31664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.022s	sys 0.025s
W20260709 12:24:47.058562 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 645946us
W20260709 12:24:47.048738 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.024s	sys 0.015s
I20260709 12:24:47.015012  1535 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:47.060154 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.024s	sys 0.015s
I20260709 12:24:47.051219 30987 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
W20260709 12:24:47.059288  1675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.019s	sys 0.026s
I20260709 12:24:47.082031 31136 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:47.082528 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:47.098496 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:47.097999  1681 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.969s	user 0.004s	sys 0.006s
I20260709 12:24:47.076376  1606 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:24:47.101450 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1831) took 1295 ms. Trace:
I20260709 12:24:47.101786 31060 rpcz_store.cc:276] 0709 12:24:45.805842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.806010 (+   168us) service_pool.cc:224] Handling call
0709 12:24:45.806389 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.806627 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.806665 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.820140 (+ 13475us) log.cc:844] Serialized 26 byte log entry
0709 12:24:45.955214 (+135074us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:45.955249 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.955313 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.023107 (+1067794us) spinlock_profiling.cc:243] Waited 9.57 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:47.100383 (+ 77276us) raft_consensus.cc:1685] finished
0709 12:24:47.100404 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.101391 (+   987us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22003456,"thread_start_us":58244,"threads_started":1,"wal-append.queue_time_us":117072}
I20260709 12:24:47.105549  1644 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:47.079851 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:47.082048  1535 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:47.087899  1566 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:47.088153  1675 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:47.106560 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.109932  1606 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:47.113193 31694 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:47.115170 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:47.115917 31690 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:47.114483  1535 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.105186  1359 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.104789  1521 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.145395 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:47.148353  1487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.010s	sys 0.055s
W20260709 12:24:47.159168  1487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.010s	sys 0.056s
W20260709 12:24:47.178037 31696 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.001s	sys 0.012s
W20260709 12:24:47.191110 31696 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.001s	sys 0.012s
I20260709 12:24:47.194072 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.195205 31695 raft_consensus.cc:2410] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:47.196177 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.199149 31691 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:47.150615  1535 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:47.198954  1683 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.798s	user 0.010s	sys 0.004s
I20260709 12:24:47.226262 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.227712 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:47.171881  1684 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.768s	user 0.012s	sys 0.000s
W20260709 12:24:47.198453  1327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.007s	sys 0.041s
I20260709 12:24:47.160274  1487 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:47.188709  1234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.012s	sys 0.026s
I20260709 12:24:47.168205  1643 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:47.277756 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.244529  1694 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.267200  1234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.012s	sys 0.027s
I20260709 12:24:47.300335 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.300997 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:47.264298  1327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.011s	sys 0.044s
I20260709 12:24:47.302399 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.266883  1716 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:47.248030  1683 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:47.291805  1474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.004s	sys 0.013s
I20260709 12:24:47.278365  1643 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:24:47.286639  1640 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.044s	sys 0.055s
I20260709 12:24:47.168756  1633 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.249614  1684 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.150724  1675 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:47.306347 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.305675 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:47.277869 31059 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.007s	sys 0.020s
I20260709 12:24:47.328852 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:47.336942 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:47.539450 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 233742us
I20260709 12:24:47.327335 31689 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.528136 31690 raft_consensus.cc:2368] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 9.
I20260709 12:24:47.541388 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:47.538372 31693 raft_consensus.cc:2368] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:24:47.543370 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.544437 31693 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:24:47.507148 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.547634 31059 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.007s	sys 0.030s
I20260709 12:24:47.549207 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.545833 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:47.465232 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.553066 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:47.553758 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:47.456677 30988 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.000s	sys 0.018s
W20260709 12:24:47.556087 30988 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.006s	sys 0.029s
I20260709 12:24:47.556581 31693 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 5: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:47.557632 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:47.560537 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.494489 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:47.558235 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.568858 31695 raft_consensus.cc:2410] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:47.558291 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:47.572190 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:47.458758 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2151) took 1351 ms. Trace:
I20260709 12:24:47.459965 30991 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:47.578763 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:47.544266 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.580396 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:47.453754 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 460540us
W20260709 12:24:47.666452 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 86407us
I20260709 12:24:47.662721 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.786401 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:47.770539 31696 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.016s	sys 0.025s
I20260709 12:24:47.717710  1263 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.368805  1709 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.301937  1694 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:47.848512  1683 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:47.848748  1709 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:47.717449  1327 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:47.722586  1701 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:47.689690  1699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.000s	sys 0.033s
I20260709 12:24:47.850160  1701 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.368012  1359 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:47.850031  1716 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.665338  1463 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.851846  1463 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:47.723788  1715 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:47.423475  1712 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.005s	sys 0.031s
I20260709 12:24:47.378762  1521 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:47.850878  1701 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:47.853605  1640 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.048s	sys 0.088s
I20260709 12:24:47.662961 31689 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:47.864238  1463 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:47.721843  1698 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.144s
I20260709 12:24:47.368922  1487 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.721184  1644 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:47.578495 30992 rpcz_store.cc:276] 0709 12:24:46.107007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.108626 (+  1619us) service_pool.cc:224] Handling call
0709 12:24:46.108979 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.245587 (+136608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.245619 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.246188 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.128286 (+882098us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b180044aee0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:24:47.457278 (+328992us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:47.457421 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.457634 (+   213us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.457785 (+   151us) raft_consensus.cc:1685] finished
0709 12:24:47.457830 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.458470 (+   640us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3347,"raft.queue_time_us":149911,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7016,"spinlock_wait_cycles":6004096,"thread_start_us":460342,"threads_started":2,"wal-append.queue_time_us":1125963}
W20260709 12:24:47.853693  1712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.010s	sys 0.056s
W20260709 12:24:47.850450  1699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.012s	sys 0.033s
I20260709 12:24:47.872341  1715 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:47.873754  1561 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:47.874547 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:47.874735 31698 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:24:47.870883 31696 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.021s	sys 0.026s
I20260709 12:24:47.719868  1686 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:47.720609  1585 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:47.848855  1694 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:47.719316  1537 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:47.368407  1620 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.889918  1717 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:47.874641  1698 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:47.890971  1646 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:47.891780 31691 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:47.892409 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:47.892681  1715 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:47.894342  1644 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:47.880509  1585 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.876865  1327 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:47.927847  1434 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.854558  1263 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.931674 30985 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.926501  1585 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:47.942588 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:47.934115  1716 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:47.947410 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1947) took 1255 ms. Trace:
I20260709 12:24:47.369606  1705 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.958074  1434 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.958985  1714 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.959863  1714 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:47.961148  1434 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:47.962047  1717 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:47.964900  1717 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:47.965386 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:47.966717 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 478) took 1583 ms. Trace:
I20260709 12:24:47.968144 31697 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:24:47.968540 31697 pending_rounds.cc:85] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 5
I20260709 12:24:47.968753 31697 pending_rounds.cc:107] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:47.967173 31696 rpcz_store.cc:276] 0709 12:24:46.384042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.384160 (+   118us) service_pool.cc:224] Handling call
0709 12:24:46.395870 (+ 11710us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:46.396220 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.396248 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:46.397642 (+  1394us) log.cc:844] Serialized 52 byte log entry
0709 12:24:47.191564 (+793922us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:47.919785 (+728221us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.952216 (+ 32431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.966200 (+ 13984us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.966318 (+   118us) raft_consensus.cc:1685] finished
0709 12:24:47.966352 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.966619 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39201,"prepare.queue_time_us":209120,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":20652,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":11600,"thread_start_us":1414097,"threads_started":2,"wal-append.queue_time_us":967930}
I20260709 12:24:47.969097 31697 raft_consensus.cc:2889] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.949285  1561 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:47.980664 31691 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:47.945927  1698 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.013290  1703 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:48.042274  1703 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.017302 31694 raft_consensus.cc:2368] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
W20260709 12:24:48.017916 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 125593us
W20260709 12:24:48.026427  1724 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:24:48.001757  1716 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:48.029685 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.033612  1651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.002s	sys 0.005s
I20260709 12:24:47.976408  1327 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:48.034612 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 133354us
I20260709 12:24:48.035128  1643 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.037088 30992 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:47.952322  1646 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.037206  1358 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:48.039652 31696 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:47.976482  1714 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:47.955866  1474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.015s	sys 0.021s
I20260709 12:24:48.053844 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:47.293815  1569 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:48.100023 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.952416 30988 rpcz_store.cc:276] 0709 12:24:46.692395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.767181 (+ 74786us) service_pool.cc:224] Handling call
0709 12:24:46.769159 (+  1978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.769347 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:47.556758 (+787411us) log.cc:844] Serialized 14 byte log entry
0709 12:24:47.886280 (+329522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:47.908776 (+ 22496us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:47.934419 (+ 25643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.940257 (+  5838us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:47.947348 (+  7091us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31712,"prepare.queue_time_us":621151,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":7628,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":13426,"thread_start_us":1068046,"threads_started":2}
W20260709 12:24:48.066335 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.123072 31692 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:24:48.180274 31059 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.011s	sys 0.031s
I20260709 12:24:48.071370 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1945) took 1379 ms. Trace:
W20260709 12:24:48.195109 31059 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.011s	sys 0.031s
I20260709 12:24:48.143844 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2136) took 3618 ms. Trace:
W20260709 12:24:48.200595 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.195567 30986 rpcz_store.cc:276] 0709 12:24:46.691928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.759052 (+ 67124us) service_pool.cc:224] Handling call
0709 12:24:46.766828 (+  7776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.022959 (+256131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.070744 (+1047785us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54003467d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:48.071157 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587355008}
W20260709 12:24:48.199155  1651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.006s	sys 0.017s
I20260709 12:24:48.197496 31061 rpcz_store.cc:276] 0709 12:24:44.525708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.161143 (+635435us) service_pool.cc:224] Handling call
0709 12:24:45.568801 (+407658us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b5800129398. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:24:45.568976 (+   175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.843615 (+1274639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.122253 (+1278638us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400268c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:48.143541 (+ 21288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2976589952}
I20260709 12:24:48.071381  1705 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.189040 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 476) took 1813 ms. Trace:
I20260709 12:24:48.054612  1684 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:48.056398  1263 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.186404  1358 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:48.216806 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:48.218485 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:48.215149 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:48.218015 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:48.216480 30958 leader_election.cc:336] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.597s (SENT)
I20260709 12:24:48.054869  1561 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.168s
I20260709 12:24:48.224819  1698 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.225498  1716 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.226446 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:48.227819  1716 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:48.231417 31693 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:48.233424 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 675) took 1977 ms. Trace:
I20260709 12:24:48.233858 31059 rpcz_store.cc:276] 0709 12:24:46.256391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.309544 (+ 53153us) service_pool.cc:224] Handling call
0709 12:24:46.311739 (+  2195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.548424 (+1236685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.548467 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.549307 (+   840us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.195710 (+646403us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:48.195749 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.206798 (+ 11049us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.233152 (+ 26354us) raft_consensus.cc:1685] finished
0709 12:24:48.233173 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.233342 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6979,"raft.queue_time_us":98313,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":11527,"thread_start_us":1864518,"threads_started":2,"wal-append.queue_time_us":667284}
I20260709 12:24:48.236395 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:48.236478  1743 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:48.237762 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:48.239007 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 846) took 1043 ms. Trace:
I20260709 12:24:48.239415 31695 rpcz_store.cc:276] 0709 12:24:47.195218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.569730 (+374512us) service_pool.cc:224] Handling call
0709 12:24:47.570498 (+   768us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:47.571622 (+  1124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.571661 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:47.572741 (+  1080us) log.cc:844] Serialized 52 byte log entry
0709 12:24:47.937218 (+364477us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:47.937348 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.937550 (+   202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.238730 (+301180us) raft_consensus.cc:1685] finished
0709 12:24:48.238749 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.238928 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58353,"raft.queue_time_us":133983,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":386,"thread_start_us":361358,"threads_started":1,"wal-append.queue_time_us":637180}
I20260709 12:24:48.241829 31695 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:48.243624 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 479) took 1859 ms. Trace:
W20260709 12:24:48.249132 31141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1687) took 3356 ms (client timeout 2139 ms). Trace:
I20260709 12:24:48.246011 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:48.247339 31698 rpcz_store.cc:276] 0709 12:24:46.384333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.384451 (+   118us) service_pool.cc:224] Handling call
0709 12:24:46.395960 (+ 11509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.886517 (+1490557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.243285 (+356768us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400690354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:48.243564 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3353992576}
I20260709 12:24:48.217520  1646 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:48.249604 31141 rpcz_store.cc:269] 0709 12:24:44.892953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.897145 (+  4192us) service_pool.cc:224] Handling call
0709 12:24:48.248756 (+3351611us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54001e91d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:48.248902 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005898624}
I20260709 12:24:48.216490  1648 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.266041 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1692) took 2319 ms. Trace:
I20260709 12:24:48.266686 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:48.267009  1701 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.267556 30986 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:48.269462  1701 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.269680 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:48.272157 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:48.273288 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:48.276211  1668 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:48.276672  1668 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:48.279053 31697 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:48.280113 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:48.280794 31697 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:48.281617 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:48.211102 31692 rpcz_store.cc:276] 0709 12:24:46.377232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.377878 (+   646us) service_pool.cc:224] Handling call
0709 12:24:47.965011 (+1587133us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:48.188945 (+223934us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9847552}
I20260709 12:24:48.282914 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:48.283679 31060 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:48.283645 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:48.285331 31692 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:48.285686 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:48.286372 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:48.287883 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:48.288175 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:48.288808 30989 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:48.289117 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:48.290752 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:48.291479 31068 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:48.292235 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1954) took 1530 ms (client timeout 1726 ms). Trace:
W20260709 12:24:48.292651 30987 rpcz_store.cc:269] 0709 12:24:46.762181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.770497 (+  8316us) service_pool.cc:224] Handling call
0709 12:24:48.291857 (+1521360us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400312c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:48.292037 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569481984}
I20260709 12:24:48.295604 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2142) took 3221 ms. Trace:
I20260709 12:24:48.296028 31062 rpcz_store.cc:276] 0709 12:24:45.073675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.187612 (+113937us) service_pool.cc:224] Handling call
0709 12:24:46.866641 (+1679029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.867004 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:46.867044 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.867101 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:46.867152 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.867227 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.295246 (+1428019us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54001bb050. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:48.295426 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18439,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3169,"spinlock_wait_cycles":3860760960}
I20260709 12:24:48.297945 30988 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:48.290108 31067 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:48.274165 30986 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:48.300256 30989 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:48.262130  1686 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:48.302639  1531 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.266536 31140 rpcz_store.cc:276] 0709 12:24:45.924957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.925099 (+   142us) service_pool.cc:224] Handling call
0709 12:24:45.925353 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.846833 (+921480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.244425 (+1397592us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b540029e5d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:48.244714 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2089292672}
I20260709 12:24:48.303035  1686 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.305433  1686 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:48.306964 30988 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:24:48.307335  1585 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:48.308988 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 255305us
I20260709 12:24:48.313444  1327 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:48.314078 30992 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.314663  1659 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:48.315282  1659 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.316258 31697 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:48.317423  1659 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:48.346146 30992 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:48.257313  1703 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:48.380733 31692 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:48.394368 30985 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:48.277835  1668 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.442888 31137 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
W20260709 12:24:48.456291 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.458021  1569 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.459223  1620 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.459484  1668 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.460189  1620 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.462921  1531 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.463584  1531 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.465082  1563 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.466125   981 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.466894   981 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.469549   981 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:48.465873  1474 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.470960  1474 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.471275  1509 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:48.465849  1563 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.527954 31060 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:48.363901  1585 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:48.663465 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 191188us
W20260709 12:24:48.748008 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.664973  1620 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:48.749771 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.751224 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.751570 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.751861 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:48.798151 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 408216us
I20260709 12:24:48.798733 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:48.586440  1563 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.823169 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:48.823626 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:48.823917 31067 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.826368  1493 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:48.826620  1531 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:48.827687 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:48.828308 31690 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.833369  1569 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:48.833987  1569 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.836856 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2150) took 2730 ms. Trace:
I20260709 12:24:48.837219 30990 rpcz_store.cc:276] 0709 12:24:46.106681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.107824 (+  1143us) service_pool.cc:224] Handling call
0709 12:24:46.108139 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:48.293698 (+2185559us) spinlock_profiling.cc:243] Waited 7.01 ms on lock 0x7b18002d7000. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb88cbfb 00007f2eeb89764b
0709 12:24:48.293902 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.293927 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:48.304319 (+ 10392us) log.cc:844] Serialized 52 byte log entry
0709 12:24:48.423805 (+119486us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:48.423839 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.751669 (+327830us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.836450 (+ 84781us) raft_consensus.cc:1685] finished
0709 12:24:48.836468 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.836657 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80,"raft.queue_time_us":140917,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":15392,"spinlock_wait_cycles":16130304,"thread_start_us":282673,"threads_started":2,"wal-append.queue_time_us":525776}
I20260709 12:24:48.838927  1493 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:48.839409  1493 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.840083  1684 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.840866  1684 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.839327  1567 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.841903  1585 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:48.750851  1643 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.852717  1567 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:48.851841 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.860375 31068 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:24:48.861677 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 59346.8us
I20260709 12:24:48.861413 31690 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:48.865948 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:48.858944  1585 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.869829 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:48.871666  1674 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.663189  1358 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:48.875097  1674 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.875910  1358 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:48.877678  1674 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:48.878679 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:48.869323  1707 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:48.870061  1585 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:48.883194 31697 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:48.888571  1684 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:48.878777  1474 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:48.890539 31661 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:48.900418 31067 raft_consensus.cc:2410] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 12:24:48.890862  1674 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:24:48.906522 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:48.907289 31059 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:48.910053  1567 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:48.881029  1707 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:48.914750  1707 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.915824  1707 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:48.919806 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.920848 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.921327 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.921842 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.923736 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 848) took 1661 ms. Trace:
W20260709 12:24:48.925468 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.924388 31693 rpcz_store.cc:276] 0709 12:24:47.262198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.851870 (+589672us) service_pool.cc:224] Handling call
0709 12:24:47.852452 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.231875 (+379423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.923429 (+691554us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540034fdd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:48.923663 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865758592}
I20260709 12:24:48.924268 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:48.937808  1674 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:48.943225 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:48.945582 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61251.1us
I20260709 12:24:48.956354 31140 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:48.951222  1674 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:48.959512 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:48.959342  1674 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:48.960857 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:48.961568 31058 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:48.966516 31059 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.975203 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:48.977079 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:48.973364  1683 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.990181 31058 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:48.964825  1493 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.003568 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.006722 31141 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:49.009687 31140 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:24:49.019465  1721 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:49.020125  1721 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.022761 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:49.023404 31144 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:49.025082  1493 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.025991 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.026757 31141 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:49.027801 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:49.003499  1683 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.030504 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 1942) took 2339 ms. Trace:
I20260709 12:24:49.030910 30991 rpcz_store.cc:276] 0709 12:24:46.691093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.757937 (+ 66844us) service_pool.cc:224] Handling call
0709 12:24:47.459123 (+701186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.579215 (+120092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.030303 (+1451088us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b540014e6d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:49.030444 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582929280}
I20260709 12:24:49.032308  1683 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:49.033296 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 209780us
W20260709 12:24:49.033905  1736 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.997s	user 0.009s	sys 0.015s
I20260709 12:24:49.034966 31106 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:49.035518 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.037628 31067 raft_consensus.cc:2410] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:49.035609 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:49.037925 31058 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:49.037569 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.026845 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.042486  1493 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.043159  1493 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.051499 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.051822 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.044703 31695 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:49.052667 31136 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:49.053174  1493 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.053803  1642 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.053876 31141 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:49.054733 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.054553 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.055116 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.055873  1758 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:49.062391  1657 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:49.063282 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.062239  1758 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.065399  1758 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.058655 31135 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:49.071403 31059 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:49.084009 31144 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:49.050057  1659 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.087459 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.086414 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.091569 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:49.093420 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.090758 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.097041 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2152) took 2862 ms. Trace:
I20260709 12:24:49.097447 31065 rpcz_store.cc:276] 0709 12:24:46.234588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.234713 (+   125us) service_pool.cc:224] Handling call
0709 12:24:46.250770 (+ 16057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.917051 (+666281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.086311 (+2169260us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400383954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:49.096870 (+ 10559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1519772544}
W20260709 12:24:49.097975  1757 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:49.098371 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.105028 31064 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:49.097039  1741 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:49.102514 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.094184 31140 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:49.105830 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.091192  1657 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.087352  1659 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:49.120965  1741 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.150898 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.151657 31144 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:24:49.157982 31695 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:49.161818  1246 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.162252 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 79071.3us
I20260709 12:24:49.162369  1246 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.164345 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:49.165102  1246 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.165887 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.166359 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.164474 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:49.167537 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.167989 31690 raft_consensus.cc:2368] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:24:49.168114 31695 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:49.168938 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.169473  1474 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.169706 31109 leader_election.cc:336] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.126s (SENT)
I20260709 12:24:49.170912 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.169875 31692 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:49.147274  1659 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.178905 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.179176 31059 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:49.149613  1631 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:49.178005 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:49.184145 31065 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:49.146687  1758 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.171334  1721 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.182865 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.193148 31695 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:49.197052  1759 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:24:49.197314  1758 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:49.200464 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90637.1us
I20260709 12:24:49.201997  1246 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.202363  1756 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.203075 31033 connection.cc:256] RPC call timeout handler was delayed by -1.1495835300002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
I20260709 12:24:49.202996  1756 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.203543  1642 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:49.204218  1642 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.206475  1642 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:49.206560 30887 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 198) took 22095 ms (client timeout 15000 ms). Trace:
I20260709 12:24:49.206737  1718 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.207333  1643 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:49.208127  1643 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.209236  1672 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:49.202752  1246 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.210415  1327 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:49.210425  1643 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.210949  1327 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:49.211907  1246 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.212185 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 674) took 3003 ms. Trace:
I20260709 12:24:49.211979  1327 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.212577 31063 rpcz_store.cc:276] 0709 12:24:46.208914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.210773 (+  1859us) service_pool.cc:224] Handling call
0709 12:24:46.238588 (+ 27815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.986466 (+747878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.211900 (+2225434us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54001a72d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:49.212059 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704395520}
I20260709 12:24:49.213490  1327 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:49.176124  1493 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.213953 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.197096 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:49.225741 31143 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:49.179090  1358 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:49.226591  1358 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.226848 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.227665 31061 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:49.228929  1358 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.207398  1718 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:49.229916  1739 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.192s	user 0.003s	sys 0.000s
W20260709 12:24:49.230012 30955 connection.cc:256] RPC call timeout handler was delayed by -1.3962051250000513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 12:24:49.230252 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1939) took 2548 ms (client timeout 2059 ms). Trace:
I20260709 12:24:49.230777 30987 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 12:24:49.230643 30984 rpcz_store.cc:269] 0709 12:24:46.681919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.682153 (+   234us) service_pool.cc:224] Handling call
0709 12:24:49.229819 (+2547666us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b54000235d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:49.230040 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":804217600}
I20260709 12:24:49.231280  1718 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.229470  1358 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.230743 30955 connection.cc:256] RPC call timeout handler was delayed by -1.1605828130000191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
I20260709 12:24:49.229395 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.170151  1474 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:49.235502 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.236186 31067 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:49.235605 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:49.237829 31068 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.237766 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.238471 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:49.241443 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.241914 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.241868 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 126982us
W20260709 12:24:49.242543 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.152690  1657 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.236784 31063 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:49.250497 31059 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:49.253623 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 461) took 4130 ms. Trace:
I20260709 12:24:49.221871  1631 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:49.258534 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:49.255235 31142 rpcz_store.cc:276] 0709 12:24:45.084254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.134179 (+ 49925us) service_pool.cc:224] Handling call
0709 12:24:46.299926 (+1165747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.300177 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.300218 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:46.300278 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:46.300333 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.300406 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.170503 (+2870097us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001ea5d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:49.214666 (+ 44163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2680192768}
I20260709 12:24:49.236965  1756 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.259608 31693 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:49.260334 31061 raft_consensus.cc:2410] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:49.260342 31058 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:24:49.261204 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:49.262367 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.215597  1563 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:24:49.266042 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.266782  1563 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:49.267163  1563 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:49.268054  1563 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.269423  1563 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:49.270802  1631 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.263590 31695 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:24:49.265337  1755 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:24:49.268473  1761 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:49.264353  1493 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.285712 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2141) took 3491 ms. Trace:
W20260709 12:24:49.206997 30887 rpcz_store.cc:269] 0709 12:24:27.110991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.118581 (+  7590us) service_pool.cc:224] Handling call
0709 12:24:27.423027 (+304446us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b5400002210. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee87700b2 00007f2ef16e9759 00007f2ef1681a76 00007f2ef1652d58 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc
0709 12:24:49.206287 (+21783260us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:39.222676 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:39.242915 (+ 20239us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:39.242968 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:24:39.242987 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:39.244032 (+  1045us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:39.244110 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:39.244135 (+    25us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:24:39.244923 (+   788us) tablet.cc:701] Row locks acquired
0709 12:24:39.244972 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:24:39.245296 (+   324us) write_op.cc:270] Start()
0709 12:24:39.245472 (+   176us) write_op.cc:276] Timestamp: P: 1783599879245215 usec, L: 0
0709 12:24:39.245498 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:24:39.294051 (+ 48553us) log.cc:844] Serialized 3570 byte log entry
0709 12:24:39.464992 (+170941us) op_driver.cc:464] REPLICATION: finished
0709 12:24:39.687194 (+222202us) write_op.cc:301] APPLY: starting
0709 12:24:39.687286 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:39.687512 (+   226us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:39.687638 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:39.849160 (+161522us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:39.849595 (+   435us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:24:39.849669 (+    74us) write_op.cc:312] APPLY: finished
0709 12:24:39.850135 (+   466us) log.cc:844] Serialized 71 byte log entry
0709 12:24:39.850372 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:39.850529 (+   157us) write_op.cc:454] Released schema lock
0709 12:24:39.850618 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:39.851067 (+   449us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:39.851104 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5035,"reported_tablets":27,"spinlock_wait_cycles":6360320,"child_traces":[["op",{"apply.queue_time_us":204397,"apply.run_cpu_time_us":4179,"apply.run_wall_time_us":163857,"num_ops":9,"prepare.queue_time_us":189618,"prepare.run_cpu_time_us":27253,"prepare.run_wall_time_us":193379,"raft.queue_time_us":23229,"raft.run_cpu_time_us":33922,"raft.run_wall_time_us":103468,"replication_time_us":219253,"thread_start_us":258128,"threads_started":4,"wal-append.queue_time_us":132773,"wal-append.run_cpu_time_us":11203,"wal-append.run_wall_time_us":1628733}]]}
I20260709 12:24:49.286052 30983 rpcz_store.cc:276] 0709 12:24:45.794015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.794459 (+   444us) service_pool.cc:224] Handling call
0709 12:24:45.794722 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.840039 (+1045317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.285281 (+2445242us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000230d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:49.285542 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2393781504}
I20260709 12:24:49.286770  1493 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.287348  1493 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.288188 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.288779 31060 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:49.288507  1761 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.289685 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:49.277750 31065 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
W20260709 12:24:49.278859 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 109321us
I20260709 12:24:49.283536 31690 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
W20260709 12:24:49.284353  1491 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.242s	user 0.005s	sys 0.025s
I20260709 12:24:49.294977 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.286655 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:49.296571 31068 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:49.296757 31692 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:49.297845 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.298573 31063 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:49.299726 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.299871  1758 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.256541  1358 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.303264  1358 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.290815  1761 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.297986 31060 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:49.304198 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.306396 31143 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:24:49.287801  1493 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.307811 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:49.304023  1491 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.254s	user 0.005s	sys 0.026s
I20260709 12:24:49.313350 31068 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:49.321820 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.324765  1585 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.327081  1585 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.327818 31060 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:49.330705  1474 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.333829 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.334242 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.335032 31062 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:49.335747 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 686) took 1017 ms. Trace:
I20260709 12:24:49.336158 31062 rpcz_store.cc:276] 0709 12:24:48.318394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.375947 (+ 57553us) service_pool.cc:224] Handling call
0709 12:24:49.334068 (+958121us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b5800129398. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876feee 000000000036eb39 00007f2ef07e31e0 00007f2ef08ac0c4 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:24:49.335675 (+  1607us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4438016}
I20260709 12:24:49.335054 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.338965 31068 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:49.311028  1707 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:49.340150 31068 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:49.340297  1707 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.341418 31068 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 12:24:49.342427 31137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 1977) took 4164 ms (client timeout 1597 ms). Trace:
W20260709 12:24:49.342907 31137 rpcz_store.cc:269] 0709 12:24:45.177891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.247609 (+ 69718us) service_pool.cc:224] Handling call
0709 12:24:49.342253 (+4094644us) spinlock_profiling.cc:243] Waited 3.17 s on lock 0x7b540001bb54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:49.342364 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7294800768}
I20260709 12:24:49.334688 31065 raft_consensus.cc:2410] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:24:49.344756 31068 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:49.348099  1531 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.349267 31109 leader_election.cc:336] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.173s (SENT)
I20260709 12:24:49.354802  1509 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:49.355436  1509 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.357414  1509 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:49.378767  1531 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.381060  1531 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:49.385263 31065 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:49.386106  1493 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.349848 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:49.406828 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.476069 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.476557 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.477173 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.412983 31693 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:49.363121 30955 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:49.480971 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.491677 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.499596 31062 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:24:49.499094  1493 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:49.500815 30983 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:49.501169 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.501760 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.499250 31137 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:49.504423 31143 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:49.501981 30984 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:49.514595  1563 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:49.515643 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 203195us
I20260709 12:24:49.515834 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:49.516320 31109 connection.cc:256] RPC call timeout handler was delayed by -2.3153664150004261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
W20260709 12:24:49.516669 31109 connection.cc:256] RPC call timeout handler was delayed by -2.3138452560001497s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4611s for a response
W20260709 12:24:49.516927 31109 connection.cc:256] RPC call timeout handler was delayed by -2.2685664150003504s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
W20260709 12:24:49.517477 31109 connection.cc:256] RPC call timeout handler was delayed by -2.1173664150005607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2379s for a response
W20260709 12:24:49.517810 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 172804us
I20260709 12:24:49.518754 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.519295 31136 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:49.517805 31109 connection.cc:256] RPC call timeout handler was delayed by -1.9698547680004594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 12:24:49.520613 31109 connection.cc:256] RPC call timeout handler was delayed by -1.84655476800026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16490000000000002s for a response
I20260709 12:24:49.520767 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:49.517812 31141 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:49.521368 31136 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.500058 31062 pending_rounds.cc:85] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:24:49.522922 31062 pending_rounds.cc:107] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:49.523159 31062 raft_consensus.cc:2889] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:49.520929 31109 connection.cc:256] RPC call timeout handler was delayed by -1.623606738000035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 12:24:49.523602 31109 connection.cc:256] RPC call timeout handler was delayed by -1.5215664150005068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30410000000000004s for a response
W20260709 12:24:49.523994 31109 connection.cc:256] RPC call timeout handler was delayed by -1.5180547680001837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
W20260709 12:24:49.524369 31109 connection.cc:256] RPC call timeout handler was delayed by -1.3911547680004333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
W20260709 12:24:49.524748 31109 connection.cc:256] RPC call timeout handler was delayed by -1.3758547680004085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
I20260709 12:24:49.470651 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 858) took 1980 ms. Trace:
I20260709 12:24:49.525934 31691 rpcz_store.cc:276] 0709 12:24:47.489935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.926938 (+437003us) service_pool.cc:224] Handling call
0709 12:24:47.940888 (+ 13950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.009838 (+ 68950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.470329 (+1460491us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b5400511b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:49.470564 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76405632}
I20260709 12:24:49.529173 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:49.533691 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:49.536201 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.536386 30984 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:49.536881 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.537184 31142 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: [2.5-5.6]
W20260709 12:24:49.537310 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.537715 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.541025 31062 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:49.542637 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.526857  1509 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.531145  1741 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.536873  1531 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.544673 30983 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:49.553334 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.554690 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:49.554093 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:49.557132 30990 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:49.528362  1563 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.561705 31058 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:49.569321 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 360760us
W20260709 12:24:49.580199 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.584380 30984 pending_rounds.cc:85] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 2
I20260709 12:24:49.584769 30984 pending_rounds.cc:107] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:24:49.585291 30991 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:49.558281 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:49.589354 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:49.591096 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:49.592957 31058 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:49.562906 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:49.594383  1747 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:49.594321 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57456.9us
I20260709 12:24:49.595046  1747 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.595448  1747 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.595890 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:49.596642  1707 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.550820  1493 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:24:49.597621 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:49.598978  1657 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:49.556164  1509 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.621833  1509 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.622845 30983 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:49.623773 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:49.624101 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.624838  1620 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:49.625306 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.626550 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.627713 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:49.628996 30989 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:49.628827  1651 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:49.629871  1684 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:49.630275  1684 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:49.602509 31137 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:49.602885  1531 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.604710 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:49.605050 31694 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 851) took 2341 ms (client timeout 2126 ms). Trace:
W20260709 12:24:49.605640 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 61250.3us
I20260709 12:24:49.606133  1761 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:49.608943 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 56308.8us
I20260709 12:24:49.662392 30990 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:24:49.605837 31689 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:49.609942  1493 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:49.610960  1563 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.615439 31664 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:49.616185  1659 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:49.691401 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:49.594767 30985 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:49.616307  1418 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.700536 31136 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:49.617130 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:49.585019 30984 raft_consensus.cc:2889] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.632382 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:49.594169 31058 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:49.684562 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:49.633687 31694 rpcz_store.cc:269] 0709 12:24:47.263141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.304354 (+ 41213us) service_pool.cc:224] Handling call
0709 12:24:49.604727 (+2300373us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b540055c654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:49.604938 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":922959616}
I20260709 12:24:49.631212  1684 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:49.702242 31142 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:49.625340 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:49.704036 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.683771 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:49.688927 30991 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:49.690104  1563 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:49.693650 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:49.693840 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.692469  1358 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:49.618047  1707 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:24:49.631862 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 167210us
I20260709 12:24:49.676297 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.715953 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:50.046121 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
W20260709 12:24:50.047345 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 356031us
W20260709 12:24:49.824707 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.047258 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 431891us
W20260709 12:24:50.045217 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 428171us
I20260709 12:24:49.989611 31058 pending_rounds.cc:85] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 3
I20260709 12:24:50.052336 31058 pending_rounds.cc:107] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:24:49.934644 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.050863 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 425353us
W20260709 12:24:50.054152 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.054908 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.052681 31058 raft_consensus.cc:2889] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:50.047214 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:50.059608 30983 raft_consensus.cc:2376] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8. Re-sending same reply.
W20260709 12:24:50.047181 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.050127 30990 raft_consensus.cc:2410] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:24:50.051499 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:50.055675 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.083319 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.900440 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:50.093050 31067 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:50.099139 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 525) took 2413 ms. Trace:
W20260709 12:24:50.094206 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.692487  1651 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:50.103564 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2173) took 1069 ms. Trace:
I20260709 12:24:50.062053 30988 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:49.848111 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:50.104071 31067 rpcz_store.cc:276] 0709 12:24:49.033720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.089047 (+1055327us) service_pool.cc:224] Handling call
0709 12:24:50.092863 (+  3816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.103343 (+ 10480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.103513 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:50.105042 30985 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 12:24:50.105265 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.077700 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.676764  1474 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:50.104178 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:50.078799 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:50.051358 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.109238 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:49.716239  1493 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:50.103391 30987 rpcz_store.cc:276] 0709 12:24:47.685403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.293640 (+608237us) service_pool.cc:224] Handling call
0709 12:24:49.230528 (+936888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.231394 (+   866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.091088 (+859694us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54000235d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:50.099065 (+  7977us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153766784}
I20260709 12:24:49.704538  1509 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:50.120700 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.047189  1684 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.109385  1761 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:50.089424  1715 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.129364 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.122887  1700 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:24:50.136377 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.123664 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.124147 30986 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 4: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:50.137070 30989 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:50.138240 31064 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:50.124442 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:49.896234  1631 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:49.749576  1693 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:50.079506  1620 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:50.128414  1702 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:50.128861 30990 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:50.129329  1633 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.130841  1358 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:50.131511 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 137171us
I20260709 12:24:49.736263  1643 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.133205 30992 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:49.765100  1758 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.354s
I20260709 12:24:50.134485 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:50.135854 31067 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:50.123514 30985 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:50.137001 31143 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:50.139557 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:50.140594 30987 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:50.121223  1651 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:50.142611 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:50.154692 31065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.003s	sys 0.027s
W20260709 12:24:50.155591 31062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.010s	sys 0.021s
I20260709 12:24:50.141531  1563 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:50.144562  1707 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:50.158196 31065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.003s	sys 0.027s
I20260709 12:24:49.737011  1491 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.413s
W20260709 12:24:50.158589 31062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.010s	sys 0.021s
I20260709 12:24:50.159449 31697 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:50.144212  1659 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:50.163662 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 486473us
I20260709 12:24:50.165473 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 490) took 1078 ms. Trace:
I20260709 12:24:50.164109 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.166643 31143 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:24:50.156558  1643 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.155628  1418 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:50.161201 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.165839 31697 rpcz_store.cc:276] 0709 12:24:49.087014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.131545 (+1044531us) service_pool.cc:224] Handling call
0709 12:24:50.165416 (+ 33871us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:50.165849 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:50.168406 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 61548.8us
I20260709 12:24:50.155788  1358 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:50.170815 31067 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:50.148625  1715 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.171900 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:50.175187 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.176805 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2175) took 1138 ms. Trace:
I20260709 12:24:50.177217 31067 rpcz_store.cc:276] 0709 12:24:49.034357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.166917 (+1132560us) service_pool.cc:224] Handling call
0709 12:24:50.170651 (+  3734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.172586 (+  1935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.172782 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:50.178330 31143 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:50.179847 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.162343  1776 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:50.152439  1702 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:50.181001 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.167948  1531 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:50.178655 31143 pending_rounds.cc:85] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 5
I20260709 12:24:50.188530 31143 pending_rounds.cc:107] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:50.188829 31143 raft_consensus.cc:2889] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:50.169344  1643 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.171231  1620 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:50.173991  1700 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.164726  1707 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.197490  1657 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:50.198763 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 692) took 1229 ms. Trace:
I20260709 12:24:50.199266 31062 rpcz_store.cc:276] 0709 12:24:48.969799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.541550 (+571751us) service_pool.cc:224] Handling call
0709 12:24:49.551271 (+  9721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.557755 (+  6484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.557782 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.618640 (+ 60858us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.164067 (+545427us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:50.164316 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.164383 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.198551 (+ 34168us) raft_consensus.cc:1685] finished
0709 12:24:50.198570 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.198707 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":45845,"thread_start_us":545073,"threads_started":1,"wal-append.queue_time_us":563170}
I20260709 12:24:50.215873 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.216645 31142 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:50.198172  1700 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:50.197906  1657 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:50.176039  1493 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.222378  1700 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.215077  1758 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:50.205075  1620 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:50.216084  1491 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:50.234863 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.242580 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:50.242450 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.243461 31143 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:50.231333  1758 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:50.227871  1418 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 12:24:50.247997 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57721.7us
I20260709 12:24:50.238554  1474 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:24:50.249792  1672 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.248139 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:50.251123  1684 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:50.258441  1672 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:50.265123  1672 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.267300  1657 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.268558  1657 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.258615  1684 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.231261  1531 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:24:50.252140  1758 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.216687  1651 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:50.283226  1747 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.687s	user 0.002s	sys 0.008s
I20260709 12:24:50.277663 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.278800 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.279814  1786 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:50.237334  1358 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.285610  1702 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:50.286453  1715 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:50.238746  1783 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:24:50.270331  1635 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:24:50.272261  1684 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.294947  1758 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.295990 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.296605 31141 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:50.296902 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.294983 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.298545  1707 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:50.298821 31140 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:50.328999 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.329659 31140 raft_consensus.cc:2376] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9. Re-sending same reply.
I20260709 12:24:50.329799 31141 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:50.331584 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2172) took 1298 ms. Trace:
I20260709 12:24:50.332224 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.332832 31140 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:24:50.333314 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.333629 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.333909 31061 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
W20260709 12:24:50.318446 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 239705us
W20260709 12:24:50.324414 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 149288us
I20260709 12:24:50.334640 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.334707 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.355918 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:50.338726  1693 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:50.364275  1693 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:50.364531 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 62998.1us
I20260709 12:24:50.365111  1693 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.366360  1693 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:50.366487 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:50.367991  1787 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.368425  1787 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.368747  1787 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.370401  1474 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:50.370775  1474 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:50.371528  1758 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:50.376502 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 112413us
I20260709 12:24:50.339794  1761 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:50.381597  1761 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:50.378132 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:50.382500  1569 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:50.382723 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.383060  1569 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:50.383591  1569 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:50.383988  1789 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:50.385186  1631 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:50.385926 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:50.386831  1641 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:50.387339  1641 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:50.386997  1672 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.386847 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:50.388263  1641 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.389377  1687 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.389782  1687 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:50.390146  1687 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.391964  1761 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.393169  1761 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:50.394989 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.395736 31061 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:50.396560 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.384570 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.343335 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 697) took 1288 ms. Trace:
W20260709 12:24:50.346100 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 166298us
I20260709 12:24:50.349818 30889 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 9, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 9 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:50.300048 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:50.399040 30958 connection.cc:256] RPC call timeout handler was delayed by -1.0054549090000364s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2002s for a response
I20260709 12:24:50.342206 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.352669 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2158) took 1623 ms. Trace:
I20260709 12:24:50.352725 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:50.398381 31067 rpcz_store.cc:276] 0709 12:24:49.054773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.182679 (+1127906us) service_pool.cc:224] Handling call
0709 12:24:50.182933 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.342891 (+159958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.342937 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:50.343019 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:50.343065 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.343150 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.343262 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77990,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":153963,"threads_started":1}
I20260709 12:24:50.400089 31060 rpcz_store.cc:276] 0709 12:24:48.729868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.311780 (+581912us) service_pool.cc:224] Handling call
0709 12:24:50.352473 (+1040693us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b54000c6554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:50.352602 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11565312}
I20260709 12:24:50.353633 31062 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:50.354589 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.270260  1620 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:50.358397 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.397310 31061 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:50.359359  1700 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.331943 31058 rpcz_store.cc:276] 0709 12:24:49.033426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.738411 (+704985us) service_pool.cc:224] Handling call
0709 12:24:49.845685 (+107274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.132412 (+286727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.132442 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.137982 (+  5540us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.331265 (+193283us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:50.331303 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.331366 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.331419 (+    53us) raft_consensus.cc:1685] finished
0709 12:24:50.331434 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.331528 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":171290,"threads_started":1,"wal-append.queue_time_us":176094}
I20260709 12:24:50.360605  1635 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:50.361176 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.361769 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.362203 31063 raft_consensus.cc:2410] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:50.362452 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.412310 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:50.413058 31143 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:50.354712  1531 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:50.363312 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.377396 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.363118 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:50.413879 31135 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:50.377605 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:50.381145 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.358273 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.404495 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.339401 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2165) took 1406 ms. Trace:
I20260709 12:24:50.381474 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 853) took 3076 ms. Trace:
I20260709 12:24:50.412961 31697 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:50.412369 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:50.422817 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45284us
I20260709 12:24:50.423475 30955 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:50.432912 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.423295 30889 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 8, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:50.419451 31068 rpcz_store.cc:276] 0709 12:24:48.932561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.526362 (+593801us) service_pool.cc:224] Handling call
0709 12:24:49.536170 (+  9808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.549854 (+ 13684us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:49.633007 (+ 83153us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.222237 (+589230us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b1800359e00. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:24:50.339076 (+116839us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:50.339127 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:50.339166 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.339247 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.339347 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78511,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":9028,"raft.queue_time_us":73,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":2963712,"thread_start_us":116403,"threads_started":1,"wal-append.queue_time_us":675836}
I20260709 12:24:50.442006 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:50.313340  1747 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.443470 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.412299  1635 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:50.371575  1474 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.447714 31136 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:50.447824 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.443351 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:50.448994 31698 raft_consensus.cc:2376] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8. Re-sending same reply.
I20260709 12:24:50.449522 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:50.404031  1631 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:50.414044 30983 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:50.450380 30955 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.409958 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:50.446645 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.448462 31060 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:50.452047 31067 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.448369  1474 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.447715 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.453845 31139 raft_consensus.cc:2410] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:50.453737 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:50.454195 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.454342 31140 raft_consensus.cc:2376] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4. Re-sending same reply.
I20260709 12:24:50.454547 31064 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:50.455353 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2176) took 1149 ms. Trace:
I20260709 12:24:50.455924 31061 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:50.456465  1563 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:50.456930  1563 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.459021  1796 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:50.450382 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:50.454432 31137 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.464609 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:50.464969 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 112289us
I20260709 12:24:50.465397 31143 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:50.460121 30889 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 8, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 8 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:50.455669 31064 rpcz_store.cc:276] 0709 12:24:49.305725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.443556 (+1137831us) service_pool.cc:224] Handling call
0709 12:24:50.454259 (+ 10703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.455061 (+   802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.455294 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:50.467163 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.469475 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.469714  1787 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.470386  1758 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.470839  1758 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:50.471230  1758 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.472107  1787 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:50.442901  1358 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:50.441726  1668 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.472783 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:50.475663 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:50.479890 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:50.477867 30889 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:50.480875 31059 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
W20260709 12:24:50.480875 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 77564.4us
I20260709 12:24:50.384532  1569 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:50.489662 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 105244us
I20260709 12:24:50.463135  1789 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:50.473553  1631 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:50.477254  1358 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.480419  1700 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:50.498306  1588 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:50.491364  1531 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.469717  1643 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.448302  1635 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.506492 30889 catalog_manager.cc:5697] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 6, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:50.497888  1796 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:50.494755  1788 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:50.502533  1358 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:50.424036 31689 rpcz_store.cc:276] 0709 12:24:47.304619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.864548 (+559929us) service_pool.cc:224] Handling call
0709 12:24:49.605569 (+1741021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.685347 (+ 79778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.377255 (+691908us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b540055c650. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:50.381393 (+  4138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4002923776}
I20260709 12:24:50.499841  1418 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:50.511310 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 46431.3us
I20260709 12:24:50.498680  1631 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:50.490790  1668 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.503955  1672 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:50.526445 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.529556 31139 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:50.538113 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:50.537792 30889 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:50.519794  1641 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:24:50.547161 30889 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 4, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 4 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:50.544154 31067 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
W20260709 12:24:50.561424 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.511091  1657 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:24:50.577528 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.630764 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.497299 31143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:50.542928  1631 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.535436  1789 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:50.519376  1531 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.567444 30889 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 8, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 8 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:50.546792  1802 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.153s
W20260709 12:24:50.656621 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.656754 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 118728us
I20260709 12:24:50.536806  1418 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:50.537915  1491 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:24:50.566054  1635 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.678980 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:50.667753 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 119770us
I20260709 12:24:50.631727  1672 raft_consensus.cc:2764] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:50.666807  1463 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 12:24:50.716777 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.719087 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 225730us
W20260709 12:24:50.726454 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.739255 31143 raft_consensus.cc:606] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:50.739660 31143 raft_consensus.cc:614] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:50.719600 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 190981us
W20260709 12:24:50.745985 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.747406 30889 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:50.679841  1418 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.630451  1783 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.080s
W20260709 12:24:50.757880 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 78983.1us
I20260709 12:24:50.569283  1588 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.159s
W20260709 12:24:50.762944 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 226000us
I20260709 12:24:50.570996  1803 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.184s
I20260709 12:24:50.710819  1633 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.712296  1788 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:50.771528 30889 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
W20260709 12:24:50.774753 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 230494us
I20260709 12:24:50.777307 31138 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:50.729004  1747 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:50.748203  1786 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:50.711685  1198 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 12:24:50.793280 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.715945  1563 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.774981  1659 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:50.801007 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:50.709748  1805 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:24:50.810377 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1855) took 1716 ms. Trace:
I20260709 12:24:50.810817 31062 rpcz_store.cc:276] 0709 12:24:49.093581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.220588 (+1127007us) service_pool.cc:224] Handling call
0709 12:24:50.353472 (+132884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.403785 (+ 50313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.810130 (+406345us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000c6550. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:50.810283 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281568640}
I20260709 12:24:50.812060 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:50.812752 31062 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:24:50.813908  1804 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:24:50.814419  1643 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:50.819307 31137 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:50.820173 30889 catalog_manager.cc:5697] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 5, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 5 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:50.785377  1633 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.829195  1633 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:50.829869 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:50.830366 31067 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:50.831059 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.831190  1803 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:50.831781 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.832105 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.832510 31067 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:50.832419 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:50.832723 31068 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:50.833328 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.833580 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:50.706327  1796 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:50.840718  1808 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:50.841480  1776 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:50.841998  1776 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.797531  1659 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:50.843391  1659 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.844133  1776 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:50.844348  1659 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.825192 30889 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 5, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 5 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:50.834180 31068 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:50.826246 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:50.839345 30955 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:50.835183  1687 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.848788 31068 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:50.855201 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 85951us
I20260709 12:24:50.815747  1672 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:50.859817 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:50.863263 31068 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:50.865149 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.866920 31060 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:50.869081 30889 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 6, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:50.849102  1783 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:50.850950  1643 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.848187  1816 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:50.876652 31109 leader_election.cc:400] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:24:50.877346 31109 leader_election.cc:403] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:24:50.870663  1633 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.860735 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:50.886339 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:50.891443 30889 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 4, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 4 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:24:50.862133  1493 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:24:50.819270  1789 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.877064  1820 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:50.883762 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 66551.3us
I20260709 12:24:50.899930 30889 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 5, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 5 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:50.902132 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:50.877239  1816 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:50.890199  1531 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:50.924103  1776 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:50.903088  1820 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.911475 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:50.926815 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:50.902915  1801 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:24:50.928983  1783 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:50.936054  1783 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.936390  1783 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.880023  1643 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:50.892447  1809 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
W20260709 12:24:50.933023  1781 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.695s	user 0.000s	sys 0.014s
I20260709 12:24:50.934537  1814 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
W20260709 12:24:50.952805 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 205296us
I20260709 12:24:50.957690 31060 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:50.945823  1531 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:50.955451  1672 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:50.996227  1531 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:50.996416  1672 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:50.996675  1531 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.997459  1672 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.938508  1820 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:51.006325 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.957578 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:50.965619  1631 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.006958  1786 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:24:51.007156 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49652.2us
I20260709 12:24:51.007476  1786 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:51.008059 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:51.008562  1631 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:51.009044  1747 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:51.009420  1747 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.009749  1747 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.011529  1651 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:51.011905  1651 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:51.012694  1651 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.013841  1651 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:51.006345 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.017808  1788 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:51.018204  1788 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:51.018632  1788 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:51.025168  1820 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:50.967417 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2167) took 1999 ms. Trace:
I20260709 12:24:50.967602  1816 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.029366 31065 rpcz_store.cc:276] 0709 12:24:48.967855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.530557 (+562702us) service_pool.cc:224] Handling call
0709 12:24:49.547578 (+ 17021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.630369 (+ 82791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.630395 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.630863 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.163271 (+532408us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:50.163295 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.163335 (+    40us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.848954 (+685619us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:50.967151 (+118197us) raft_consensus.cc:1685] finished
0709 12:24:50.967171 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.967362 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":3898112,"thread_start_us":532063,"threads_started":1,"wal-append.queue_time_us":560521}
I20260709 12:24:51.018186  1747 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:50.973994  1758 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.001s
I20260709 12:24:51.095805  1758 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.097750  1758 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:50.974581  1687 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:50.980854  1693 consensus_meta.cc:287] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.000s
I20260709 12:24:51.065394  1803 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:50.985966 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2174) took 1952 ms. Trace:
I20260709 12:24:51.098999 31059 rpcz_store.cc:276] 0709 12:24:49.034034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.089997 (+1055963us) service_pool.cc:224] Handling call
0709 12:24:50.118705 (+ 28708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.481934 (+363229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.985757 (+503823us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54000c7454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:50.985911 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832496000}
W20260709 12:24:50.986747 31063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 690) took 2017 ms (client timeout 2518 ms). Trace:
W20260709 12:24:50.988201  1761 consensus_meta.cc:287] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
I20260709 12:24:51.100005  1747 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:50.991520 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:50.994813  1684 consensus_meta.cc:287] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.635s	user 0.006s	sys 0.002s
I20260709 12:24:50.990289 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1856) took 1868 ms. Trace:
I20260709 12:24:51.102043  1684 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.102366 31058 rpcz_store.cc:276] 0709 12:24:49.121405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.422852 (+1301447us) service_pool.cc:224] Handling call
0709 12:24:50.988581 (+565729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.988954 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:50.988993 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:50.989049 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:50.989086 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.989144 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.990096 (+   952us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b54001bab50. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:50.990207 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3271,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":1240620800}
I20260709 12:24:50.931479  1491 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:24:51.104166  1684 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:50.995766 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:51.028674  1657 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:51.105383  1657 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:51.105427 31062 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:51.106303  1657 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.107244  1493 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:51.107542  1657 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:51.107709  1493 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:51.108773  1493 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.028540  1788 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.109973  1493 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.082343  1830 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:51.111099  1788 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:51.036407  1776 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:51.082907  1683 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:51.111829  1776 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.984324  1418 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:51.086619 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 229872us
I20260709 12:24:51.113102 31135 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:51.113842 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:51.060343  1820 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:51.115023  1820 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:51.116211 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:51.061468  1787 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:51.116879  1787 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:51.059489 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
W20260709 12:24:51.062384 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.984325 31061 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:51.118719 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:50.966928  1817 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.118501 31142 raft_consensus.cc:606] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:51.066256  1641 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:24:51.118638 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.119565  1758 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:51.119961  1641 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:51.119645  1817 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.120900  1641 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.121368 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:50.910600 30889 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 1 to 3, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 3 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:24:51.071149  1643 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:51.091804 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 83810.5us
I20260709 12:24:51.122249  1817 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:51.100049 31063 rpcz_store.cc:269] 0709 12:24:48.969198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.299404 (+330206us) service_pool.cc:224] Handling call
0709 12:24:50.986511 (+1687107us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001bab54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:50.986671 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441274240}
I20260709 12:24:51.083364  1707 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:24:51.122061  1641 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.123494 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:51.123957  1707 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:51.124058 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:51.124552  1707 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:51.125079  1828 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:51.125429  1707 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.126276 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:51.127000  1828 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:51.127869 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:51.128060 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:51.118481  1687 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:51.119760 31142 raft_consensus.cc:614] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:51.129084  1828 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:51.047070  1358 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:51.094853 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 183482us
W20260709 12:24:51.095046 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 234874us
I20260709 12:24:50.983446 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:51.139528 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 156169us
I20260709 12:24:51.139767 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:51.140460 31106 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:51.140604  1358 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:51.141250  1358 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:51.141592  1358 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.141887  1358 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:51.142478  1806 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:51.142971  1806 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.143296  1806 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.146378  1509 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:51.146767 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:51.147110 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:51.147367 31059 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:51.146801 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:51.150724 31067 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:51.150905 31063 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.151763  1789 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:51.147711  1509 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:51.154379  1509 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:51.155062  1588 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:51.155618  1588 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.155937  1509 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.157548  1588 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:51.160471 31059 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:24:50.954600  1563 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:51.031448  1668 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:51.029860  1816 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.164004 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:51.165141 31106 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:51.110971 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:51.165913 31062 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:51.165782  1563 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:51.167623  1809 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:51.167964  1809 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:51.168752  1809 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.169910  1809 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:51.113211 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.142547 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:51.172533  1659 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:51.172900  1659 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.173000 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:51.173278  1659 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.174181 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:51.175071 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:51.176443 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:51.177038 31135 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:24:51.117702  1787 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.177791  1474 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:51.178205  1474 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.178392  1787 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:51.178643  1474 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:51.122097  1758 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:51.074174  1786 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.111840  1830 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:51.129555  1828 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:51.189178  1830 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:51.131897 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:51.084280 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:51.132926  1776 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.134287 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:51.135497 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:51.136096 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:51.136811 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:51.195818 31690 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.127888  1643 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:51.163148 31063 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:24:51.229480 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 97634.5us
I20260709 12:24:51.064412 31068 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:24:51.230405 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:51.230883 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:51.231889 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:51.232478 31068 pending_rounds.cc:85] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 6
I20260709 12:24:51.232874 31068 pending_rounds.cc:107] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:24:51.233168 31068 raft_consensus.cc:2889] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:51.188208  1786 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:51.188742  1588 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.190562 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 106369us
I20260709 12:24:51.193130 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.193310 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:51.081794  1531 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.242131 31141 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.959947  1569 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:51.100910  1803 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:51.229595 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.230429 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:51.171695 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:51.181685 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.234481 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:51.261584 31068 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:51.235984 31698 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:51.237069 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:51.069063  1633 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:51.244201 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:51.247301 31135 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:51.249136 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:51.214201 31062 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:51.250159 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:51.252010 31144 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:51.252925 31065 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:51.235837  1567 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:51.240664  1758 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:51.268153 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.272370 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:51.273335 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:51.277263 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:51.257411 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:51.267122 31060 pending_rounds.cc:85] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 7
I20260709 12:24:51.296454 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.243789  1668 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.250943  1198 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:51.189862  1828 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.088001  1783 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.254211  1588 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.243494  1830 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.299005  1788 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.498s
I20260709 12:24:51.797115  1835 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:51.797731  1788 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:51.798182  1788 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:51.295063  1567 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.799041 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:51.799417 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:51.799741 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:51.800081 30983 raft_consensus.cc:2410] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:51.800360 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:51.441270 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:51.331638 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:51.801819 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:51.802246 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:51.802624 30984 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:51.749398  1659 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.576s	user 0.003s	sys 0.005s
I20260709 12:24:51.802891 30985 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:51.803303  1659 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.804558  1817 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.804723  1588 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:51.805083  1817 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.805383  1588 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:51.805586  1659 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:51.735280  1531 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:51.258924  1803 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:51.330359 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:24:51.747728  1809 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.577s	user 0.002s	sys 0.005s
I20260709 12:24:51.713436  1573 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.508s
I20260709 12:24:51.807142  1817 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:51.807014  1803 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.807621  1573 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:51.807989  1573 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:51.808049  1803 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.275710  1707 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.330051 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:51.750684  1776 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:51.809146 31689 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:51.674929 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.335937 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.813213 31135 raft_consensus.cc:2410] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:24:51.613273 31690 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:51.678817 31696 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:51.808154 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.329893 31691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:24:51.815817 31137 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.811723 31060 pending_rounds.cc:107] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:24:51.816579 31060 raft_consensus.cc:2889] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:51.811223 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 553873us
W20260709 12:24:51.811297 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 538042us
W20260709 12:24:51.802389 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 470827us
I20260709 12:24:51.818907 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:51.791232 31141 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.000s
I20260709 12:24:51.821197 31141 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
W20260709 12:24:51.823709 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.826539 31142 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:51.827159 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.524320 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:51.828068 31692 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.403522 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:51.830417 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:51.831097 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.251024  1820 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:51.654024 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:51.835839 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.842718 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:51.845304 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:51.330459 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.846246 30992 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:51.256645  1569 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:51.851071 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:51.799741  1567 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.802321  1830 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:51.291915  1672 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:51.864267 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.869424 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:51.246462  1643 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:51.871696  1569 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.872272  1852 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:51.872776 31063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.005s	sys 0.005s
I20260709 12:24:51.872850 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:51.873824 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:51.900115 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.900421 31135 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:51.876338 31141 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:24:51.878399 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 606070us
I20260709 12:24:51.881875 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:51.828766  1758 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.839641  1657 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:51.834527  1776 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:51.884342 31143 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:51.808341  1668 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.292131  1761 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.581s
I20260709 12:24:51.330166  1463 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.543s
I20260709 12:24:51.840226  1814 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:51.845115  1588 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:24:51.901870 31063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.006s	sys 0.005s
I20260709 12:24:51.904026 31137 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:51.907799 31138 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:51.943992 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:51.952958 31059 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:51.945341 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:51.879676  1418 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:51.799063  1788 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.876788  1635 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:51.962765 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:51.907323  1474 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.001s
I20260709 12:24:51.909292  1758 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:51.968278 30983 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 4.
I20260709 12:24:51.978147 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:51.980209 30984 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:51.948622  1659 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.982548  1659 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.983307  1474 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:51.985116 31136 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:51.978979 31138 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:51.989269 31138 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:51.989729 31138 pending_rounds.cc:85] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 3
I20260709 12:24:51.990154 31138 pending_rounds.cc:107] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:24:51.990465 31138 raft_consensus.cc:2889] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:51.977794  1635 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.995558 30984 raft_consensus.cc:2410] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:24:51.989542  1758 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:51.999517  1668 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:51.904651  1573 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.001044  1643 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:52.004989 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.057736 31142 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:52.065172 31692 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
W20260709 12:24:52.066300 31106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.022s	sys 0.046s
W20260709 12:24:52.006008  1845 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 12:24:51.967648  1852 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:52.045372  1815 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.520s	user 0.003s	sys 0.005s
W20260709 12:24:52.045904  1816 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.882s	user 0.006s	sys 0.000s
W20260709 12:24:52.044466  1841 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.765s	user 0.003s	sys 0.000s
W20260709 12:24:52.029306  1493 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.919s	user 0.008s	sys 0.000s
I20260709 12:24:52.076093 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:52.111140 30984 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:52.029747  1788 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.027079  1835 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:52.071561  1358 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.929s	user 0.005s	sys 0.005s
I20260709 12:24:52.005954  1758 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:51.955101  1814 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.062078 31139 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
W20260709 12:24:52.092501 31106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.022s	sys 0.046s
I20260709 12:24:52.180274 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:52.180660 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.106469  1852 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:52.104809 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:52.185110 30983 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
W20260709 12:24:52.089406  1641 consensus_meta.cc:287] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.966s	user 0.011s	sys 0.000s
W20260709 12:24:52.111847 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 293177us
I20260709 12:24:52.137636 30992 raft_consensus.cc:2410] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:52.070693  1588 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.174798  1493 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.178352 31142 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:52.178930 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:52.198211 31138 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:24:52.181298 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.182144 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.182328 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:52.094436 31030 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.000s	sys 0.026s
W20260709 12:24:52.207940 31030 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.000s	sys 0.026s
I20260709 12:24:52.180881 30984 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
W20260709 12:24:52.195473 31693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.000s	sys 0.013s
I20260709 12:24:52.192232 30987 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
W20260709 12:24:52.209458 31693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.000s	sys 0.013s
W20260709 12:24:52.201773 31068 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.001s	sys 0.013s
I20260709 12:24:52.179567  1418 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:24:52.195307  1509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.001s	sys 0.012s
W20260709 12:24:52.206895 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.014s	sys 0.012s
W20260709 12:24:52.210899 31068 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.002s	sys 0.013s
W20260709 12:24:52.221855 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.015s	sys 0.012s
W20260709 12:24:52.191896  1687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.006s	sys 0.006s
I20260709 12:24:52.211254 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:52.212301 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:52.205904 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 304453us
W20260709 12:24:52.171214 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 298413us
I20260709 12:24:52.188539  1852 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:52.222754 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.210351  1817 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:52.182372  1358 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.230532 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:52.403335 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.517815 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:52.249735 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.433735 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 798) took 1159 ms. Trace:
I20260709 12:24:52.248288 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1881) took 1102 ms. Trace:
I20260709 12:24:52.479151 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2196) took 1288 ms. Trace:
W20260709 12:24:52.484061 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.232167 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1880) took 1086 ms. Trace:
W20260709 12:24:52.520190 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 280257us
W20260709 12:24:52.475409 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 288789us
W20260709 12:24:52.413609 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.440730 31689 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.001s
I20260709 12:24:52.527109 31689 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:52.523097 31067 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:52.527781 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 801) took 1233 ms. Trace:
I20260709 12:24:52.528245 31689 rpcz_store.cc:276] 0709 12:24:51.294390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.298503 (+  4113us) service_pool.cc:224] Handling call
0709 12:24:52.527704 (+1229201us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:52.248426 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.068s	user 0.013s	sys 0.015s
I20260709 12:24:52.517433 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:52.521330 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.528403 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 97542.7us
I20260709 12:24:52.201939  1849 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.522133 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.529660 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.029s	sys 0.033s
W20260709 12:24:52.221746  1509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.001s	sys 0.012s
I20260709 12:24:52.541635 31691 raft_consensus.cc:606] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:52.542273 31691 raft_consensus.cc:614] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:52.520699 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:52.294276 31065 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:52.523095 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.535012 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 1.35434e+06us
I20260709 12:24:52.521739 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:52.521178 31058 rpcz_store.cc:276] 0709 12:24:51.145753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.162453 (+ 16700us) service_pool.cc:224] Handling call
0709 12:24:51.162758 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.162939 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.162963 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.163386 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.282406 (+119020us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:51.282443 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.282510 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.248027 (+965517us) raft_consensus.cc:1685] finished
0709 12:24:52.248064 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.248218 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":118632,"threads_started":1,"wal-append.queue_time_us":116902}
I20260709 12:24:52.521196 31068 rpcz_store.cc:276] 0709 12:24:51.191213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.271776 (+ 80563us) service_pool.cc:224] Handling call
0709 12:24:51.272031 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.272206 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:52.222029 (+949823us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.478515 (+256486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:52.478633 (+   118us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:52.478714 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.478845 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.479081 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30807,"prepare.queue_time_us":455099,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":36,"raft.queue_time_us":126,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":3507,"thread_start_us":954468,"threads_started":2}
I20260709 12:24:52.520910 31693 rpcz_store.cc:276] 0709 12:24:51.274046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.274556 (+   510us) service_pool.cc:224] Handling call
0709 12:24:51.274849 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.275079 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.275103 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.303366 (+ 28263us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.210073 (+906707us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:52.210131 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.210553 (+   422us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.433484 (+222931us) raft_consensus.cc:1685] finished
0709 12:24:52.433510 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.433669 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":556079,"threads_started":1,"wal-append.queue_time_us":982239}
I20260709 12:24:52.541127 30984 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:52.520406 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:52.542742 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:52.528461 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:52.518592  1857 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.475732 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2197) took 1284 ms. Trace:
I20260709 12:24:52.550804 31062 rpcz_store.cc:276] 0709 12:24:51.191470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.278013 (+ 86543us) service_pool.cc:224] Handling call
0709 12:24:51.278303 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.278504 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:52.110782 (+832278us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.469742 (+358960us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:52.475233 (+  5491us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:52.475437 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.475517 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.475663 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":244685,"prepare.queue_time_us":449038,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":69554,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":9396,"thread_start_us":721440,"threads_started":2}
W20260709 12:24:52.229009  1687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.007s	sys 0.006s
I20260709 12:24:52.431339  1851 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.363159  1829 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.206100  1493 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.547459  1814 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:52.554396 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.540506  1569 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.000s
I20260709 12:24:52.556160 31059 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:52.521765 31061 rpcz_store.cc:276] 0709 12:24:51.145448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.148309 (+  2861us) service_pool.cc:224] Handling call
0709 12:24:51.148606 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.148946 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.148975 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.149400 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:24:51.279131 (+129731us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:51.279173 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.279237 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.044103 (+764866us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b240015a028. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee862d9c4 00007f2ee862bacb 00007f2ee87e6dc9 00007f2ee87e6a55 00007f2ee87e609d 00007f2eeb41810e 00007f2eeb42f88c 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221
0709 12:24:52.231851 (+187748us) raft_consensus.cc:1685] finished
0709 12:24:52.231881 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.232110 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15150,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"spinlock_wait_cycles":52769664,"thread_start_us":129268,"threads_started":1,"wal-append.queue_time_us":129700}
I20260709 12:24:52.227443  1418 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.389169  1783 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:52.569638 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 47874.6us
I20260709 12:24:52.539628  1643 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:52.577982 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:52.560014  1835 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:52.560003  1829 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.586476 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:52.578975  1862 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.579789  1850 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:52.226871  1855 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.347s
I20260709 12:24:52.556401  1816 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:52.589836 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 58238.1us
I20260709 12:24:52.594743 31138 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:24:52.594929 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:52.594277  1783 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.572903  1569 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.552047  1849 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.602043  1816 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.609565  1835 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:52.560984  1857 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.591085  1761 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:24:52.589548  1643 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.571655  1851 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:52.635453  1803 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.003s
I20260709 12:24:52.635639 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2194) took 1445 ms. Trace:
I20260709 12:24:52.650188 30983 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:52.650545  1787 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.457s	user 0.006s	sys 0.007s
I20260709 12:24:52.599256  1862 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.673913 31064 rpcz_store.cc:276] 0709 12:24:51.190628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.232227 (+ 41599us) service_pool.cc:224] Handling call
0709 12:24:51.276067 (+ 43840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.295409 (+ 19342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.295437 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.418772 (+123335us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.138520 (+719748us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:52.138557 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.138621 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.635408 (+496787us) raft_consensus.cc:1685] finished
0709 12:24:52.635426 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.635549 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18960,"raft.queue_time_us":19055,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":102078,"thread_start_us":321496,"threads_started":1,"wal-append.queue_time_us":904833}
I20260709 12:24:52.626396  1816 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:52.699120 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.652876  1859 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:52.679512  1855 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:52.622797  1783 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.672122  1872 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:24:52.672835  1874 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:52.676442  1835 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:52.688788  1803 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.709164  1806 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.565s	user 0.012s	sys 0.001s
I20260709 12:24:52.719379 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:52.706255  1850 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.687386  1509 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 12:24:52.739773 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 130650us
I20260709 12:24:52.708822  1358 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:52.742828 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 200141us
I20260709 12:24:52.745836 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:52.723246  1567 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.838s	user 0.008s	sys 0.003s
I20260709 12:24:52.718361  1881 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.758687 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:52.719051  1859 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:52.761307 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 50949.7us
I20260709 12:24:52.747047  1850 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.767709  1850 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:52.703670  1873 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.123s
I20260709 12:24:52.708148  1810 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:24:52.740553  1877 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:52.774330 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:52.774835 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 246467us
I20260709 12:24:52.777619 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:52.774978 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:52.772508  1563 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:52.782838  1876 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:24:52.771771  1881 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.785461  1881 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:52.764075  1567 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.787987  1567 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:52.789139  1643 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:52.791002  1851 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:52.791499  1851 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.790540  1828 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.791970  1851 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.792589  1828 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.793853  1851 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:52.778309  1418 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:52.794565  1828 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:52.795099  1828 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.795532  1828 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.799441  1849 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:52.799767  1828 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:52.803236  1862 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:52.805537  1816 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.805575 30983 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:52.808104 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1740) took 1513 ms. Trace:
I20260709 12:24:52.808601 31144 rpcz_store.cc:276] 0709 12:24:51.294868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.836815 (+541947us) service_pool.cc:224] Handling call
0709 12:24:51.837202 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.837469 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.837507 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.848998 (+ 11491us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.375291 (+526293us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:52.375342 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.375442 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.807865 (+432423us) raft_consensus.cc:1685] finished
0709 12:24:52.807889 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.808043 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":476533,"threads_started":1,"wal-append.queue_time_us":808766}
W20260709 12:24:52.819795 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 218898us
I20260709 12:24:52.790540  1835 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:52.722153  1803 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:52.724982  1806 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:52.870800 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 92857.4us
I20260709 12:24:52.794571  1493 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:52.814464  1886 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:52.873076 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.875474 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:52.852711  1783 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:52.859937  1867 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
W20260709 12:24:52.883466 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.882687 31064 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:52.895936 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:52.894254 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:52.859596  1707 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:52.897804 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:52.912616 30990 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 6: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:52.896893 31059 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:24:52.901611 31140 raft_consensus.cc:2368] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8.
I20260709 12:24:52.866077  1567 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:52.985212 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 164632us
W20260709 12:24:53.045599 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.870867  1829 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:53.055279 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.055212  1816 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:53.074510 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:53.126626 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59170 (request call id 1) took 1886 ms. Trace:
I20260709 12:24:53.127153 31691 rpcz_store.cc:276] 0709 12:24:51.256860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.256976 (+   116us) service_pool.cc:224] Handling call
0709 12:24:53.126340 (+1869364us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b54005c1a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88e853 00007f2ef085c60a 00007f2ee9de27d4 00007f2ee9de2762 00007f2ee9de2691 00007f2ee9de260c 00007f2ee9de18c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:53.126475 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1668212608}
I20260709 12:24:53.130409 31144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:53.130968 31144 raft_consensus.cc:606] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.131279 31144 raft_consensus.cc:614] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:53.134042 31144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:24:53.134579 31144 raft_consensus.cc:606] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:53.134336  1859 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.135936 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:53.138871 30991 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:53.055497 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.137885 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:53.162067 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:53.164286 31059 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:24:53.181860 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.866384  1881 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 12:24:53.055423  1747 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.887267  1509 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.260s
I20260709 12:24:53.123772 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:53.330809 31063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.003s	sys 0.007s
W20260709 12:24:53.387506 30957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.006s	sys 0.008s
W20260709 12:24:53.406039 31063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.004s	sys 0.007s
W20260709 12:24:53.406092 30957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.006s	sys 0.008s
W20260709 12:24:53.319231 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.217314 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.187736 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:53.407840 30991 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
W20260709 12:24:53.292536 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.248008 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.224244 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:53.411298 31059 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:24:53.403159 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 625567us
W20260709 12:24:53.421015 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.121994  1879 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.344s	user 0.000s	sys 0.000s
W20260709 12:24:53.429378 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 354946us
I20260709 12:24:52.888218  1569 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:52.887960  1877 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:24:53.432011 30985 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1973) took 1632 ms (client timeout 1906 ms). Trace:
I20260709 12:24:53.441351 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:53.078560  1788 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.883s	user 0.011s	sys 0.001s
I20260709 12:24:52.880245  1886 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:53.116348  1882 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.217252  1883 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.441043 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.220599  1857 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:53.208092  1871 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.443s	user 0.000s	sys 0.001s
W20260709 12:24:53.220170  1830 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.334s	user 0.011s	sys 0.001s
I20260709 12:24:53.467947 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.472412 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:53.239586  1852 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.876s	user 0.008s	sys 0.002s
W20260709 12:24:53.326987 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.005s	sys 0.005s
W20260709 12:24:53.477746 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.005s	sys 0.007s
W20260709 12:24:53.478137 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 900142us
I20260709 12:24:53.166134  1707 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:24:53.446975 30985 rpcz_store.cc:269] 0709 12:24:51.800028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.800145 (+   117us) service_pool.cc:224] Handling call
0709 12:24:53.431806 (+1631661us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5400024ed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:53.431946 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4445056}
I20260709 12:24:53.451836 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:53.452919 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.455785 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:53.454468  1897 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:53.487306 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.487942 30990 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:53.164283  1893 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:53.439489 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.456768 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:53.457430  1851 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:53.111927  1855 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:53.441710 31141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1742) took 2137 ms (client timeout 2262 ms). Trace:
I20260709 12:24:53.463037 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:53.448647  1758 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.476708 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:53.301324  1806 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:53.451099  1588 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.489137 30989 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:53.222288  1803 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:53.458282  1788 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.487672  1563 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:53.487607  1897 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:53.506669  1886 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.490777  1885 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.506929  1789 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:53.506044  1761 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.493229 31141 rpcz_store.cc:269] 0709 12:24:51.295340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.835605 (+540265us) service_pool.cc:224] Handling call
0709 12:24:53.432475 (+1596870us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b540024ac54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:53.432635 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90873088}
I20260709 12:24:53.474831  1873 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
W20260709 12:24:53.512892 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49957.1us
I20260709 12:24:53.577561 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:53.579066 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:53.522255  1830 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:53.513770 30991 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:53.461457  1882 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.436762  1747 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:53.518498 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:53.466661  1881 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:53.460397  1889 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.117s
I20260709 12:24:53.474675  1418 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:53.501595 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.602403 30985 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:53.561012 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.487147  1707 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.451268  1877 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.486280  1857 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:24:53.580451 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.513410 31059 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:53.588294 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:24:53.596979 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 116544us
I20260709 12:24:53.488853  1852 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.608211 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:53.601016 30992 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1977) took 1735 ms (client timeout 1812 ms). Trace:
I20260709 12:24:53.463814  1883 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:53.609102 30992 rpcz_store.cc:269] 0709 12:24:51.866102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.866262 (+   160us) service_pool.cc:224] Handling call
0709 12:24:53.600763 (+1734501us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54002c2254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:53.600937 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542765824}
I20260709 12:24:53.521979  1758 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.592803  1830 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:53.596542  1567 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:53.612226  1805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.765s	user 0.050s	sys 0.067s
I20260709 12:24:53.527447  1588 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:53.600606  1509 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:53.601348  1872 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:53.636201 31691 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:53.603909  1789 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.630245 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49881.4us
I20260709 12:24:53.604568  1886 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.566391  1788 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:53.650372 31033 connection.cc:256] RPC call timeout handler was delayed by -1.1071890739999617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 12:24:53.650645 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:53.650913 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0648890739998933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
W20260709 12:24:53.638377  1805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.798s	user 0.051s	sys 0.067s
W20260709 12:24:53.636457 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 184736us
I20260709 12:24:53.634796 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1892) took 1039 ms. Trace:
I20260709 12:24:53.640817 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:53.588881  1761 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.647316 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.628522  1358 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.631278  1901 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:53.586927  1885 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:53.651571 31063 rpcz_store.cc:276] 0709 12:24:52.595836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.596092 (+   256us) service_pool.cc:224] Handling call
0709 12:24:52.596397 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.596662 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:52.615378 (+ 18716us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.634372 (+1018994us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.634443 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:53.634506 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.634590 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.634726 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18689,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":2771,"raft.queue_time_us":183041,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":13091,"spinlock_wait_cycles":1403264,"thread_start_us":1012414,"threads_started":2,"wal-append.queue_time_us":982633}
W20260709 12:24:53.651230 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0333890739998424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
I20260709 12:24:53.638610  1830 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.655315 30992 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:53.655514 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.645254  1872 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:53.656764 30985 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:53.645377  1509 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.645704  1835 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:53.641987  1857 consensus_queue.cc:1243] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:24:53.667485 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0225890739998249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 12:24:53.652052 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:53.655345  1900 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:53.686980 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 210349us
I20260709 12:24:53.688905 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:53.660921  1659 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.676203  1872 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.652697  1789 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:53.714762 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 62781us
I20260709 12:24:53.689643 30992 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:53.681725  1873 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:53.695917 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.682729  1509 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:53.716997 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45014.4us
I20260709 12:24:53.668926  1358 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:53.721359 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.722741 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.725060 31141 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:53.640868  1855 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.719004 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:53.685516  1908 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:53.717572  1857 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.737565 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.734184 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:53.742466 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.744323 31139 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:53.751116 31137 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:53.752285 31137 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:53.752579 31137 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:53.754591 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:53.756045 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.756701 31138 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 2.
I20260709 12:24:53.757414 31691 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:53.740087 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:53.758445 31142 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:53.758605 31141 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.639541  1897 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.760015 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.761538 31138 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:24:53.726265  1659 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:53.718243  1914 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:53.723788  1915 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:53.766454 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:53.767136 31136 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2018) took 2518 ms (client timeout 2558 ms). Trace:
I20260709 12:24:53.767266 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.773352 31138 raft_consensus.cc:2368] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8.
I20260709 12:24:53.772959 31141 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:53.774296 31139 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:53.775018 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:53.753396 31137 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:53.772744 31136 rpcz_store.cc:269] 0709 12:24:51.248665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.248851 (+   186us) service_pool.cc:224] Handling call
0709 12:24:53.766957 (+2518106us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54001eafd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:53.767082 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628302208}
I20260709 12:24:53.732870  1805 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:53.777627 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:53.780021 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:53.780853 31136 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.780140 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:53.782142 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:53.783355 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:53.784360 31137 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:53.785420 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.531s (SENT) [suppressed 19 similar messages]
I20260709 12:24:53.778373 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.786084 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:53.788504 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:53.789227 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:53.748440  1788 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.776993 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.790117 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:53.767896  1873 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.762660  1889 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:53.809571 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:53.811285 31697 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:53.825492 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.826391 31141 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:53.816895 31137 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:53.776923  1835 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:53.817761  1761 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.818876  1789 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.822435  1920 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:53.828302 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:24:53.834203 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.833384 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.782678  1914 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:53.789831  1895 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.796594  1747 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:53.841775 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.790050  1901 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.822642  1919 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 12:24:53.765024  1857 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:53.836346 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1972) took 2037 ms (client timeout 1578 ms). Trace:
I20260709 12:24:53.775031  1918 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:53.849458 30987 rpcz_store.cc:269] 0709 12:24:51.799371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.799480 (+   109us) service_pool.cc:224] Handling call
0709 12:24:53.836170 (+2036690us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b5400225e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:53.836282 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901102336}
I20260709 12:24:53.835634 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:53.839195 31698 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:53.839857 31689 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 12:24:53.823486  1852 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.843745 30991 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:53.835359 31141 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:53.756320  1908 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.847259 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:53.857646 31697 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.843137 31695 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:53.822743  1904 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:53.853715 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.829757  1788 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.856909 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.857694 31690 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:53.859984 31141 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:53.860997 30992 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:53.865747 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:53.866510 30987 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.861729 31137 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:53.866041 31141 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:24:53.868438 31137 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:53.860920 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75092.9us
I20260709 12:24:53.868438 31141 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:53.869350 31141 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:53.845108  1883 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:53.844935  1758 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.850374  1588 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:53.871949 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:53.845268  1882 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.864591 30989 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
I20260709 12:24:53.853552  1835 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:53.875043  1913 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:24:53.875530 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:53.871249  1906 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:24:53.897668 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:53.875972  1885 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:53.879479 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:53.880131  1830 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.830535  1873 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.881899 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:53.882638 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:53.886639  1905 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
W20260709 12:24:53.887054  1903 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:24:53.888137  1358 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:53.871374 31137 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:53.850185  1918 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.893110 30987 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:53.894048 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.890175  1788 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.871054  1904 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:53.847975  1877 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.847309  1889 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:53.906392 31697 raft_consensus.cc:2410] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:24:53.911690 31136 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:53.913501  1923 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:53.921474 31064 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:53.938294 31694 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:53.940316 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.789439  1915 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.946017 31143 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:53.929505 30992 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:53.931255 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.909782  1852 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:24:53.937412 31140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2011) took 2765 ms (client timeout 1906 ms). Trace:
W20260709 12:24:53.953905 31140 rpcz_store.cc:269] 0709 12:24:51.172187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.173939 (+  1752us) service_pool.cc:224] Handling call
0709 12:24:53.924259 (+2750320us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540006fed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:53.937342 (+ 13083us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3956452608}
I20260709 12:24:53.941825 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:53.943670 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.936925  1918 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.928202 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:53.949529  1921 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:53.950289  1908 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.841167  1920 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.950428 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.952505  1915 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.955111 30992 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:24:53.983932 30992 pending_rounds.cc:85] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 5
I20260709 12:24:53.984228 30992 pending_rounds.cc:107] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:24:53.984508 30992 raft_consensus.cc:2889] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:53.955322 30987 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:53.955585 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:53.848662  1895 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.971704 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:53.977342 31067 raft_consensus.cc:2410] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:24:53.983042 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.987679  1881 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:53.988102  1881 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.988451  1881 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:53.988497  1924 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:53.990599 30987 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:53.854372  1901 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:53.974846  1567 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:53.994064 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:53.999284 30989 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:53.944465  1747 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:24:53.994397  1921 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:54.005329  1857 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:54.007280  1904 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.007050  1872 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:54.007704  1858 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.403s	user 0.011s	sys 0.018s
I20260709 12:24:54.008250 31693 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:24:54.009341 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1886) took 2846 ms. Trace:
I20260709 12:24:54.009734 31060 rpcz_store.cc:276] 0709 12:24:51.163252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.197098 (+ 33846us) service_pool.cc:224] Handling call
0709 12:24:51.266898 (+ 69800us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:51.817418 (+550520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.817480 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:51.817980 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.183014 (+365034us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:52.183061 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.183147 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.006897 (+1823750us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:24:54.009063 (+  2166us) raft_consensus.cc:1685] finished
0709 12:24:54.009083 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.009283 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13273984,"thread_start_us":293170,"threads_started":1,"wal-append.queue_time_us":677519}
I20260709 12:24:54.011140  1830 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.011471 30985 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.011512  1920 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:53.996377  1567 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.012393  1567 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.946328  1914 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:54.013406  1846 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 2.337s	user 0.006s	sys 0.010s
I20260709 12:24:54.016376 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.016810  1901 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.017043 31061 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:54.017982  1904 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.018502  1904 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.019112 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:54.019688 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2177) took 1128 ms. Trace:
I20260709 12:24:54.020210 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:54.020705 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:54.020808 31140 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.020907 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.021543 31136 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:54.011704  1830 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.022567  1830 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.023466  1895 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:54.024505  1830 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:53.992944  1908 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.025206  1573 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:54.026108  1908 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.026466  1747 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:54.026850  1747 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:54.027290  1747 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:54.028065  1635 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:54.030649 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1985) took 1805 ms (client timeout 2024 ms). Trace:
I20260709 12:24:54.020035 30991 rpcz_store.cc:276] 0709 12:24:52.891561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.593242 (+701681us) service_pool.cc:224] Handling call
0709 12:24:53.836545 (+243303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.854854 (+ 18309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.019523 (+164669us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400225e50. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:54.019637 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558645504}
I20260709 12:24:54.021884 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:54.033214 30986 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:54.012813 30985 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 12:24:54.031062 30984 rpcz_store.cc:269] 0709 12:24:52.225052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.225234 (+   182us) service_pool.cc:224] Handling call
0709 12:24:54.030441 (+1805207us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b540022d8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:54.030590 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47673600}
I20260709 12:24:54.040578 30989 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:54.042577 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.043452 31697 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:54.045401 30991 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:54.003679  1915 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.054332 30985 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:54.051694 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.052102 30984 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:54.055667 31136 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
W20260709 12:24:54.056226 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.025225  1788 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:54.017530  1889 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.058100 30989 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:54.042923  1921 raft_consensus.cc:2764] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:54.020337  1904 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:54.060557  1923 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:54.068979 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:54.079866 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.089186 31140 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:54.090405 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.091153 31140 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:24:54.046389  1920 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:24:54.116482 31140 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:54.169919 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:54.196988 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.172021 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.168430 31060 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:54.202235 31059 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.167299 31063 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:54.176595 30988 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:24:54.201211 31058 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:54.093600 31137 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:54.168468 31061 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:54.203310 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.088570  1620 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:54.199625 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.203840  1567 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.207787 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:54.208177  1855 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:54.208518 31140 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:54.208732  1874 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:54.208859  1886 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:54.211623 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:54.212450  1872 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.213608 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.207278  1620 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.217244  1509 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:54.167929  1900 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:54.216962 31697 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:54.215205 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.151561  1474 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:54.196231  1873 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.226910 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.358076 31137 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
W20260709 12:24:54.264225 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.282014 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.231128 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.273140 31059 raft_consensus.cc:2410] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:24:54.283448 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:54.384336 31136 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.361904 31058 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:54.387432 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.226310 31144 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
W20260709 12:24:54.388880 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 181858us
I20260709 12:24:54.234193 31692 raft_consensus.cc:2368] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9.
I20260709 12:24:54.229924 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:54.392053  1915 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.398078 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:54.280956  1918 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.403077  1918 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:54.403788 31135 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:54.393203  1886 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:54.225813  1855 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.406961  1855 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:54.407136 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:54.150592  1923 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:54.407956 31137 raft_consensus.cc:2376] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7. Re-sending same reply.
I20260709 12:24:54.399135 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:54.409631 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.412802 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.415215 31144 raft_consensus.cc:2410] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:24:54.415983 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:54.416500 31144 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:24:54.416719 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:54.417542 31144 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:54.417660 31068 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:54.450474 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:54.230666  1567 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.217417  1874 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:54.462064 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.467649 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:54.468022 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.468977 31136 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:54.473011 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:54.473544 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.474939 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.475386 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.278803  1747 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:54.339355  1881 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.483728 31696 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:54.489159 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:54.464586  1758 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.513697 31140 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:54.514339 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:54.466563  1855 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:54.521574 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.518993 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:54.523579 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:54.524724 31697 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:24:54.480497  1872 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:54.527472 31068 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:54.495697  1620 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:54.514878  1493 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.548849 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.477424  1830 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:24:54.519301  1758 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:54.523023  1918 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:54.654151  1938 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
W20260709 12:24:54.695257  1936 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:24:54.520046  1855 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:54.711117  1942 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:24:54.897027 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2194) took 1254 ms. Trace:
I20260709 12:24:54.547847  1872 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.897655 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 544) took 1154 ms. Trace:
I20260709 12:24:54.835135  1747 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.894973  1918 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.897542 30992 rpcz_store.cc:276] 0709 12:24:53.643027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.988023 (+344996us) service_pool.cc:224] Handling call
0709 12:24:53.988389 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.988709 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.988737 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.989265 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.123175 (+133910us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:54.127758 (+  4583us) log.cc:844] Serialized 14 byte log entry
0709 12:24:54.136833 (+  9075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.136932 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.896804 (+759872us) raft_consensus.cc:1685] finished
0709 12:24:54.896823 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.896967 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6106,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":6240,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":5029,"spinlock_wait_cycles":1273984,"thread_start_us":133564,"threads_started":1,"wal-append.queue_time_us":409488}
I20260709 12:24:54.899444  1918 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.899868  1758 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.898252 30987 rpcz_store.cc:276] 0709 12:24:53.742771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.991327 (+248556us) service_pool.cc:224] Handling call
0709 12:24:53.991623 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.991967 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.991998 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.997927 (+  5929us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.195640 (+197713us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:54.222860 (+ 27220us) log.cc:844] Serialized 14 byte log entry
0709 12:24:54.362613 (+139753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.363418 (+   805us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.897472 (+534054us) raft_consensus.cc:1685] finished
0709 12:24:54.897489 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.897601 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48175,"prepare.queue_time_us":27331,"prepare.run_wall_time_us":36,"raft.queue_time_us":173567,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":51859,"spinlock_wait_cycles":52245888,"thread_start_us":149468,"threads_started":1,"wal-append.queue_time_us":274024}
I20260709 12:24:54.741912 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.900781 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.901423 31058 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 3.
I20260709 12:24:54.901789 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:54.779858 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.556546 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:54.860221 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.903724 31068 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:54.896107  1855 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:54.902583 31065 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
W20260709 12:24:54.655553  1945 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:24:54.557291 31696 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:24:54.913019 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 877) took 1101 ms. Trace:
W20260709 12:24:54.647773  1943 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:24:54.914294 31696 rpcz_store.cc:276] 0709 12:24:53.812114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.529486 (+717372us) service_pool.cc:224] Handling call
0709 12:24:54.556960 (+ 27474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.912691 (+355731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.912950 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.915777 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.919332 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.920114 31065 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 3.
W20260709 12:24:54.921500 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 402600us
I20260709 12:24:54.917034 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.922459 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:54.667681  1881 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:54.770812  1934 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:54.757179  1889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.114s	sys 0.172s
I20260709 12:24:54.897656  1872 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.934121  1635 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:54.913924  1928 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.922663  1631 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:54.940359 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2171) took 2050 ms. Trace:
I20260709 12:24:54.558100  1874 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:54.894941  1493 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:54.945035  1889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.119s	sys 0.177s
I20260709 12:24:54.946071 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:54.954372 31696 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:54.955591 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.957154 30986 rpcz_store.cc:276] 0709 12:24:52.889833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.891830 (+  1997us) service_pool.cc:224] Handling call
0709 12:24:54.031399 (+1139569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.039454 (+  8055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.940152 (+900698us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540022d8d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:54.940299 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2619975936}
W20260709 12:24:54.958666 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 294052us
I20260709 12:24:54.962675 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:54.952715  1635 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.966131  1809 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:54.966498  1635 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:54.968518  1493 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:54.969170  1874 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:54.959118  1928 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:54.973266  1928 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:54.953042  1620 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:24:55.002743 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:55.003026 31068 raft_consensus.cc:2410] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:55.003623 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 810) took 1180 ms. Trace:
I20260709 12:24:55.004006 31696 rpcz_store.cc:276] 0709 12:24:53.823263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.932331 (+1109068us) service_pool.cc:224] Handling call
0709 12:24:54.954104 (+ 21773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.963250 (+  9146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.003565 (+ 40315us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:55.007750 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47096.3us
I20260709 12:24:55.008786 31065 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:55.010095 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:55.009872 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48821.4us
I20260709 12:24:55.014592 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:55.015986 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:55.020083  1620 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.020516  1620 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.020859  1620 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.021195  1889 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:55.021605  1889 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.021930  1889 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:55.013835  1641 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.022524  1641 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.022847  1641 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.025046  1631 raft_consensus.cc:993] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:24:55.025467  1631 raft_consensus.cc:1081] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:24:54.950156  1934 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.028069  1934 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.029120 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2212) took 1150 ms. Trace:
I20260709 12:24:55.029511 31067 rpcz_store.cc:276] 0709 12:24:53.878806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.881663 (+  2857us) service_pool.cc:224] Handling call
0709 12:24:55.028913 (+1147250us) spinlock_profiling.cc:243] Waited 55.9 ms on lock 0x7b54000c7e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.029056 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128568576}
I20260709 12:24:55.030259  1493 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:55.034413 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.033304  1493 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.036074  1493 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.031569  1635 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.037436 31058 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:55.035106 31067 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:24:55.034835 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.049407 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.015818 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:55.041918 31065 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:24:55.049288  1758 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.052624  1758 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.055079  1951 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.055546  1951 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.055850  1951 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:55.059718 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.061685 31067 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:55.062158 31058 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:55.053335  1881 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.076500  1809 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:55.090198  1809 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.091387  1918 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.093494  1918 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.075973  1872 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.083864  1881 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:55.121147  1891 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.894s	user 0.005s	sys 0.012s
I20260709 12:24:55.135251  1872 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.136909 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:55.140280  1758 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.140686 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:55.141335 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:55.143298  1635 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.144055  1509 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:55.145231 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2179) took 2026 ms. Trace:
I20260709 12:24:55.146587  1897 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.145596 30990 rpcz_store.cc:276] 0709 12:24:53.119357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.600229 (+480872us) service_pool.cc:224] Handling call
0709 12:24:53.627886 (+ 27657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.628070 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:53.641787 (+ 13717us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.144703 (+1502916us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b1800356020. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a
0709 12:24:55.144888 (+   185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:55.144937 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:55.145002 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.145068 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.145173 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26553,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"spinlock_wait_cycles":6975872,"thread_start_us":25047,"threads_started":1,"wal-append.queue_time_us":119822,"wal-append.run_cpu_time_us":1416,"wal-append.run_wall_time_us":1108927}
I20260709 12:24:55.156617  1923 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:55.156975 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.159730 31061 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.168880 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 505) took 1407 ms. Trace:
I20260709 12:24:55.169242 31143 rpcz_store.cc:276] 0709 12:24:53.761162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.849709 (+ 88547us) service_pool.cc:224] Handling call
0709 12:24:53.938100 (+ 88391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.954875 (+ 16775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.168677 (+1213802us) spinlock_profiling.cc:243] Waited 88 ms on lock 0x7b54003f43d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.168797 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202479360}
I20260709 12:24:55.157939  1934 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:55.170732 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 803) took 1420 ms. Trace:
I20260709 12:24:55.170814  1934 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.171411  1934 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:55.172410 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 504) took 1388 ms. Trace:
I20260709 12:24:55.172762 31694 rpcz_store.cc:276] 0709 12:24:53.784149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.784847 (+   698us) service_pool.cc:224] Handling call
0709 12:24:53.785228 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.946273 (+161045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.172213 (+1225940us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54002f99d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:55.172357 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350820096}
I20260709 12:24:55.174461 31697 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:55.171147 31691 rpcz_store.cc:276] 0709 12:24:53.750258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.750478 (+   220us) service_pool.cc:224] Handling call
0709 12:24:53.750812 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.757867 (+  7055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.170508 (+1412641us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b540044a054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:55.170676 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14215040}
I20260709 12:24:55.183261 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:55.183786 31138 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:55.183911 31143 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:55.174031 31694 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:55.186489 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:55.187209 31142 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.192133 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 512) took 1107 ms. Trace:
W20260709 12:24:55.192225 30983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1986) took 2824 ms (client timeout 2077 ms). Trace:
I20260709 12:24:55.180765 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:55.192625 30983 rpcz_store.cc:269] 0709 12:24:52.368206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.368370 (+   164us) service_pool.cc:224] Handling call
0709 12:24:55.192051 (+2823681us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540029a9d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.192157 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276026752}
I20260709 12:24:55.192550 31691 rpcz_store.cc:276] 0709 12:24:54.070402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.176771 (+1106369us) service_pool.cc:224] Handling call
0709 12:24:55.177012 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.177176 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.177200 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:55.177237 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:55.177273 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.177321 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.177404 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.194414 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:55.109469  1620 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.198916 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 883) took 1360 ms. Trace:
I20260709 12:24:55.201104 31694 rpcz_store.cc:276] 0709 12:24:53.838246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.173598 (+1335352us) service_pool.cc:224] Handling call
0709 12:24:55.173878 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.193650 (+ 19772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.198843 (+  5193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.205192 31138 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:55.211748  1881 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.195933  1872 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.212538  1881 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:55.231937 31142 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:55.212208  1889 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:55.324239 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.345716 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 80749.5us
I20260709 12:24:55.199219  1747 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:55.249536  1881 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:55.392923 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 236795us
I20260709 12:24:55.393183  1961 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.393728  1961 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.396180  1961 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.396875  1961 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:24:55.382140 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 187784us
I20260709 12:24:55.373921  1531 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.233392  1872 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.409003 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:55.390749  1830 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:55.423600  1889 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.413093  1531 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.426614  1531 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.426810  1881 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.431071  1620 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:55.432716  1872 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:55.404028  1923 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:55.436110 31064 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:24:55.459581 31064 pending_rounds.cc:85] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 8
I20260709 12:24:55.459861 31064 pending_rounds.cc:107] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:24:55.460136 31064 raft_consensus.cc:2889] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.436429 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:55.452057 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.446352  1620 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.435876  1493 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.466365 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 504) took 1705 ms. Trace:
I20260709 12:24:55.466732 31139 rpcz_store.cc:276] 0709 12:24:53.760904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.768335 (+  7431us) service_pool.cc:224] Handling call
0709 12:24:53.773941 (+  5606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.774660 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.466133 (+1691473us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b5400062f50. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.466308 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12323584}
I20260709 12:24:55.467687  1493 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.437270  1641 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.455314  1928 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.470932  1641 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.470608  1928 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.456833 31061 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:55.463335  1620 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.437140  1923 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:24:55.474509  1418 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:24:55.474936  1923 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:55.436336  1855 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.000s
I20260709 12:24:55.476230  1855 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.434199  1872 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:55.478250  1855 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:55.479450 31694 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:55.480037 31698 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:55.478279  1872 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.484997  1872 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:55.455745  1934 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.486928  1620 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:55.488051  1951 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.489293 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:55.490016  1951 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:55.490403 31058 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:24:55.492264 31135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 501) took 1750 ms (client timeout 1977 ms). Trace:
I20260709 12:24:55.492705 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 811) took 1661 ms. Trace:
W20260709 12:24:55.492617 31135 rpcz_store.cc:269] 0709 12:24:53.741997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.743060 (+  1063us) service_pool.cc:224] Handling call
0709 12:24:55.492092 (+1749032us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400063954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.492207 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1436034816}
I20260709 12:24:55.482913 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 879) took 1670 ms. Trace:
I20260709 12:24:55.486987  1934 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.494563 31697 rpcz_store.cc:276] 0709 12:24:53.815120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.563311 (+748191us) service_pool.cc:224] Handling call
0709 12:24:55.174290 (+610979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.174810 (+   520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.482741 (+307931us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b540044a050. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.482858 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205780352}
I20260709 12:24:55.482901 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.497449 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 503) took 1857 ms. Trace:
I20260709 12:24:55.498077  1962 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:55.498711 31059 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:55.493156 31696 rpcz_store.cc:276] 0709 12:24:53.831093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.004752 (+1173659us) service_pool.cc:224] Handling call
0709 12:24:55.013286 (+  8534us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:55.018015 (+  4729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.018059 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:55.058910 (+ 40851us) log.cc:844] Serialized 52 byte log entry
0709 12:24:55.386501 (+327591us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:55.386553 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.386631 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.492503 (+105872us) raft_consensus.cc:1685] finished
0709 12:24:55.492524 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.492654 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":304434,"threads_started":1,"wal-append.queue_time_us":385196}
I20260709 12:24:55.499604 31141 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.501292 31693 rpcz_store.cc:276] 0709 12:24:53.640443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.640588 (+   145us) service_pool.cc:224] Handling call
0709 12:24:55.497286 (+1856698us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b540081e7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.497388 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818319744}
I20260709 12:24:55.505242 31061 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:55.505514 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.506002 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.506480 31141 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.502053  1830 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:55.501538  1962 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.507599  1830 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.507688  1962 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:55.542531  1886 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.543187  1886 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.544580 31689 raft_consensus.cc:2410] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:24:55.536404  1934 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:55.549399  1707 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.555881 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 510) took 1497 ms. Trace:
I20260709 12:24:55.561573 31698 rpcz_store.cc:276] 0709 12:24:54.058446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.472660 (+414214us) service_pool.cc:224] Handling call
0709 12:24:55.555597 (+1082937us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54006a22d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.555792 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2309608832}
I20260709 12:24:55.560943  1934 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.567317  1934 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.561338  1859 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:55.569222  1859 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:55.570535 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.570149  1859 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.571233  1859 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.572643  1358 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.573141  1358 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:55.574688 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 166486us
I20260709 12:24:55.575380 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:55.576143 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:55.576843 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:55.578538 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.578168 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.581048 30986 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:55.545985  1881 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:24:55.586748 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.586927 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.590451 30987 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.590535 30986 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:55.587494 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.588176 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.547618  1897 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 12:24:55.594370 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 103408us
I20260709 12:24:55.603497 31141 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:55.604305 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1764) took 1838 ms. Trace:
I20260709 12:24:55.601979 31061 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:55.604699 31141 rpcz_store.cc:276] 0709 12:24:53.765727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.867952 (+102225us) service_pool.cc:224] Handling call
0709 12:24:53.868291 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.499249 (+1630958us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b4c0036d840. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee9b60a20 00007f2ee9b71a29 00007f2ee9b37644 00007f2ee9b5ef2d 00007f2ee9b5eb93 00007f2eeb88f18b 00007f2eeb88fcb5 00007f2eeb88af30 00007f2eeb88c7de 00007f2eeb896b49 00007f2eeb897416
0709 12:24:55.603985 (+104736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.604238 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54005504}
I20260709 12:24:55.618844 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:55.626403 30958 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:55.626379 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.633507 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:55.634826 30987 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:55.638252 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:55.639712 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55123.4us
I20260709 12:24:55.640797 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:55.643971 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:55.646760 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 252697us
I20260709 12:24:55.647713 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:55.648726  1934 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.649214  1934 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.649638  1934 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.624131  1820 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.625375  1641 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:55.681480 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:55.625408  1896 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.685868 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:55.687747  1950 raft_consensus.cc:2764] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:24:55.689147 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:55.689867  1950 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.690229  1950 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.634276  1966 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:55.690760  1950 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:55.691133 31692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 884) took 1845 ms (client timeout 2211 ms). Trace:
W20260709 12:24:55.691509 31692 rpcz_store.cc:269] 0709 12:24:53.846302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.213325 (+367023us) service_pool.cc:224] Handling call
0709 12:24:55.690938 (+1477613us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b54005e7754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.691077 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37420800}
I20260709 12:24:55.692490  1915 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:55.693396  1920 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:55.693775  1920 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:55.694540  1920 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.695751  1920 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:55.695887 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.696938  1830 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.700700  1893 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.702863  1747 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.703516  1747 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.705683  1747 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.639335  1620 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:55.712747 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 96836.8us
W20260709 12:24:55.707662 31690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 505) took 1922 ms (client timeout 2150 ms). Trace:
I20260709 12:24:55.716668 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:55.717777 31061 raft_consensus.cc:2410] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:24:55.709563 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:55.720150 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:55.645886  1928 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.724258 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:55.725389 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.728617 31063 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:55.713896 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:24:55.714846 31690 rpcz_store.cc:269] 0709 12:24:53.784794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.785792 (+   998us) service_pool.cc:224] Handling call
0709 12:24:55.707519 (+1921727us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b54005e6ad4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.707610 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161916288}
W20260709 12:24:55.727424 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 101131us
I20260709 12:24:55.628774  1889 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:24:55.653697  1829 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:55.632050  1588 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:24:55.749367 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.758160 30984 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:55.738456 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:55.782846 31065 raft_consensus.cc:2410] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:55.686798  1869 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.731513 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:55.791447 30958 leader_election.cc:400] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8.
I20260709 12:24:55.793320 30958 leader_election.cc:403] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:24:55.791438 31060 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:24:55.783347 30987 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:55.792488 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.793998 30985 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:55.794503 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:55.796234 30986 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:55.796151 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:55.797781 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.798839 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:55.802577 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.802733  1971 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:55.803175  1830 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.803294  1971 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:55.803902  1830 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:55.804353  1830 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:55.804240  1971 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:55.805330 30988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1987) took 3375 ms (client timeout 1825 ms). Trace:
I20260709 12:24:55.805188  1830 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:55.805709 30988 rpcz_store.cc:269] 0709 12:24:52.430506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.430663 (+   157us) service_pool.cc:224] Handling call
0709 12:24:55.805181 (+3374518us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540014dcd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.805272 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3692300544}
I20260709 12:24:55.806509 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.712735  1418 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:55.726722  1896 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.814559 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.835546 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.836058 30991 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.836062 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.837877 31065 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:24:55.838932 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:55.804733  1886 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.818478  1700 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:24:55.725430  1747 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:55.767122  1965 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:55.766865  1928 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:24:55.806761 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.855953 30988 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:55.826169 31689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 878) took 2013 ms (client timeout 1612 ms). Trace:
W20260709 12:24:55.859040 31689 rpcz_store.cc:269] 0709 12:24:53.812435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.857325 (+ 44890us) service_pool.cc:224] Handling call
0709 12:24:55.811314 (+1953989us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540034fdd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:55.826077 (+ 14763us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3752276736}
I20260709 12:24:55.750798  1531 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:55.827812 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.869901 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.877549 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 236825us
I20260709 12:24:55.877758  1966 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:55.881484 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.881867 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.882611 31058 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:55.880864 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 126728us
I20260709 12:24:55.738934  1855 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 12:24:55.767589  1358 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:55.833571 31109 connection.cc:256] RPC call timeout handler was delayed by -1.5149763609997535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 12:24:55.684473  1820 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.835520  1829 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.892225  1829 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.892989  1969 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:55.738734  1968 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.148s
I20260709 12:24:55.894481  1829 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:55.895087 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.870412  1531 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.895905  1531 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:55.730299  1877 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:55.896658  1896 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:55.897228  1896 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:55.897796  1896 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.869340  1747 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.884465 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.899583  1418 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.900133 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.900054  1418 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:55.900730 30988 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:24:55.901944 30988 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:55.900348  1747 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.869143 30989 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:55.892477  1588 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:55.826714  1973 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8.
I20260709 12:24:55.840557  1869 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:55.905704 31058 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:24:55.934969 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.883486 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:55.916560 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:55.929776 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.943596 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.944463 31061 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:55.945339 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:55.936128 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 140066us
I20260709 12:24:55.937398 31058 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:55.947031 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.947108 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:55.948170 31061 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.948259 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:55.949131 31058 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:24:55.949908 31058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:24:55.950532 31058 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:55.950919 31058 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:55.952505 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73526.1us
I20260709 12:24:55.952970  1965 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:55.955116  1968 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:55.955493  1968 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:55.956337  1968 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.957477  1968 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.957552 31061 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:55.958369  1707 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:55.959198 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 75789.1us
I20260709 12:24:55.959849  1707 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.960693 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:55.961834  1707 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:55.963896 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:55.964661 31141 raft_consensus.cc:2410] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:55.965057  1975 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:55.965461 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:55.965590  1975 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:55.966176 31139 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:55.966470  1975 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.967033 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:55.967626 31138 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:55.958483 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.959198 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.968775 30988 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:55.969581 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:55.959357  1881 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.953938  1965 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.970504  1881 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.970887  1881 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.964505 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:55.971845 31142 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.974896 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:55.976116 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:55.977759 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:55.979485 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:55.980537 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:55.981237 31690 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:55.982353 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:55.985538  1707 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.986020  1707 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:55.988070  1707 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:55.991653 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.992826 31142 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:55.994621 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:55.995183 31690 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.997051  1981 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:55.999015 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:56.003180 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.003715 31692 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:56.004623 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:56.007503 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:56.005388 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.022310 31689 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
W20260709 12:24:56.008212 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 59929.8us
I20260709 12:24:56.008944  1851 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:56.013288 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.941869 31065 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:24:55.904201  1820 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:56.016245  1981 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.015020  1881 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.020494 30991 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:56.018455 31690 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:24:55.902199  1418 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:56.026706 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.028548 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:56.033244 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.046398 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.036782 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.023655 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:55.902045  1872 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.029600 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.050086 31062 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:56.051142 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.052469 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.032752  1588 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:24:56.056805 31062 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:55.905776  1896 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.057920 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:56.064989 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:56.058921 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 357561us
W20260709 12:24:56.066573 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.070063 31691 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:56.066546 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:56.030651  1474 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:24:56.023824  1877 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:56.029902  1620 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:24:56.078785 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.056967  1886 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:56.064098  1889 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:56.067873 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:56.029753  1829 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:24:56.079363 30989 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.062675 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.079973 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.020671  1869 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:56.057700  1979 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:56.033097  1707 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:56.080020 31058 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:56.094264 31061 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:56.077456 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:56.101755 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:56.103188  1829 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:56.103739  1829 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.076005  1976 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:56.106312 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:56.069875  1851 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:56.118556 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.121021 31058 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:56.074680  1855 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:56.124893 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:56.104174  1979 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:56.133651 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:56.119230  1700 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.120972  1358 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:56.121142  1829 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:56.127298  1851 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:56.152907 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 210472us
I20260709 12:24:56.157038 31058 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:56.126873  1707 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:56.128450  1855 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.165333 30989 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:56.164974  1896 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:56.132417  1641 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.166314 31060 raft_consensus.cc:2410] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:56.129253  1983 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:56.131814  1620 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:56.141536  1358 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.122643  1973 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 12:24:56.160387  1934 consensus_meta.cc:287] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.001s
I20260709 12:24:56.141669  1820 raft_consensus.cc:2764] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:56.151141  1700 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.190670  1358 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.132035  1950 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.165352  1851 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:56.064024  1966 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.169872  1855 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.159304  1889 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:24:56.214602  1358 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:56.197546 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 169066us
W20260709 12:24:56.199749 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 91885.7us
I20260709 12:24:56.218578  1965 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:56.221938  1358 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:56.223536  1700 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:56.224000  1707 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.193625  1620 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:56.171653  1977 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 12:24:56.235034 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 127693us
I20260709 12:24:56.235929 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:56.240388 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 100036us
I20260709 12:24:56.243881 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.244385 31142 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:56.194626  1934 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.252687 31141 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:56.230593  1877 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:24:56.256635 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.256807 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.254981 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.257330 31138 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:56.257520 30987 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 4.
I20260709 12:24:56.247095  1975 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:56.260667 31141 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:56.234344  1965 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:24:56.261226 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:56.284075 31065 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:56.285727 30983 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.255223 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:56.264796 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 110215us
I20260709 12:24:56.235028  1897 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:24:56.290027 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:56.290845 31067 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.237965  1974 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:24:56.284195 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.294132 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:56.258061 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:56.295933 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.296908 31141 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:56.248665  1588 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:24:56.258440  1881 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:56.288169 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:56.301295 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.304684 30987 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:56.306550 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.302654 30990 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:56.200212  1979 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:56.261023  1872 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:56.307433 30984 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:56.311491 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:56.315873 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:56.259038  1700 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:56.296497  1983 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
W20260709 12:24:56.297559  1962 consensus_meta.cc:287] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.789s	user 0.011s	sys 0.008s
I20260709 12:24:56.322074  1934 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:56.261775  1820 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:24:56.322374  1962 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:56.324162  1986 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:24:56.324580  1962 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:56.327296  1829 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:56.318938  1984 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:56.328382  1984 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:56.329059  2004 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.318589 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:56.330709 31143 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:56.318533 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:56.320758  1988 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:56.333007  2004 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:56.321086  1975 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:56.321947  1987 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:56.329648  1829 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:56.292969  1971 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:56.329188  1984 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.295924  1973 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:56.317346  1968 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.335208 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:56.313892 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.338176 31067 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:56.363794 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.364025 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.364497 30990 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:56.364849 30984 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:56.339679 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:56.340847 30983 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:56.344182  1950 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:56.317129  1358 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:24:56.345108 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.341794  1981 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:56.358116 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.358983 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.379655 31140 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:56.320655 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:56.354753 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.305805  1869 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:56.366254 31062 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:56.356302 31141 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:56.353260  1975 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.377017 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.391762 30990 raft_consensus.cc:2368] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 4. Current term is 5.
I20260709 12:24:56.354871  1872 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:56.375399 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:56.382192 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 131693us
I20260709 12:24:56.389652 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:56.390435 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.377650 31064 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:56.346328  1881 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:24:56.403215 30988 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:24:56.381696  1829 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:56.405632 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.402792 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.402896 31137 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:56.407150 31137 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:56.408100 31137 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:56.409422 31137 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.402992 31067 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:56.411581 30990 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:56.297731  1620 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.413317 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:56.413825 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 112598us
I20260709 12:24:56.377367  1973 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:56.415864 30987 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:24:56.394658  1975 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:56.414750 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.401975  1872 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.419072 31690 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:56.416751 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:56.403183  1984 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.426440  2002 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:56.427812  2002 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:56.428196  2002 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:56.428941  2002 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:56.432034 30988 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:56.435091 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:56.435220  1758 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:56.435689 30990 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.435738  1758 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:56.436915  1758 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.438448  1965 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:56.439808 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.441365 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.402410  1981 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:56.443260  1981 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:56.444141  1531 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.000s
I20260709 12:24:56.444569  1531 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.447913  2012 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:56.454490  1965 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:56.461158 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.464094 31137 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:56.401680  1869 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:24:56.499125 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:56.538066 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.593570 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:56.539383 30990 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:56.594401 31060 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:24:56.596019 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:56.596673 31060 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:56.599308 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 251723us
I20260709 12:24:56.596690 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:56.602808 30984 raft_consensus.cc:2376] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10. Re-sending same reply.
W20260709 12:24:56.557329 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 140631us
I20260709 12:24:56.605314 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.462123  1573 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:56.606711 30985 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:56.604815 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:56.590811  1758 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.604797 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:56.455893  1855 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.629019 30984 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:24:56.602569  2002 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.630396 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:56.456102  1872 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.598354  1531 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:56.631788 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:56.643077 30985 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:56.606319  1881 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.608266  1923 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:56.631387 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.654425 30988 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:56.573030  1877 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.639009 31060 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:24:56.608712  1573 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:56.657765 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.657893 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.612659  1920 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:56.665428 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.681254 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.683507 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:56.685477 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.630992  1869 consensus_queue.cc:1243] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:56.631187  2014 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.698616 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.704375 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:56.704939 30985 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.705014 31689 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:56.704504 31698 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:56.646497  2004 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:56.708392 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:56.734591 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 104376us
W20260709 12:24:56.743304 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 112967us
I20260709 12:24:56.663388  1965 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:56.746940 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.750700 31060 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:56.750867 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:56.747200 31696 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:56.598670  1973 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.683872  2021 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:56.755673 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.784034 30992 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:56.784457 31698 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:56.779472 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.792479 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.703135  1968 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:56.794595 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:56.664004  1881 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.797808 31140 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:56.746963  1869 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:56.693805  1920 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:56.797997 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.800653 30985 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:24:56.814741 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.815623 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.816414 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.817788 31140 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:56.859141 30992 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:56.860579 31143 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 4.
I20260709 12:24:56.823426 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:56.851198 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58717.6us
I20260709 12:24:56.861678 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.862613 31144 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 4: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:56.866585 31141 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:56.866986 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:56.753520  2029 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:56.871243 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:56.873238 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:56.986963 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2231) took 1054 ms. Trace:
W20260709 12:24:56.909996 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.032948 31061 rpcz_store.cc:276] 0709 12:24:55.932960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.959421 (+ 26461us) service_pool.cc:224] Handling call
0709 12:24:55.959687 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.094602 (+134915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.986659 (+892057us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54001bb054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:56.986887 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309071232}
W20260709 12:24:57.013093 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.946782 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:57.036604 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:57.038141 31136 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:56.920148  2015 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 12:24:57.038198  1881 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:56.798663  1829 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.241s
I20260709 12:24:57.039853 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:57.040455 30987 raft_consensus.cc:2376] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7. Re-sending same reply.
I20260709 12:24:57.041244 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 565) took 1083 ms. Trace:
I20260709 12:24:57.041323 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:57.041694 30989 rpcz_store.cc:276] 0709 12:24:55.957801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.958235 (+   434us) service_pool.cc:224] Handling call
0709 12:24:57.041079 (+1082844us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400226854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:57.041189 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254862592}
I20260709 12:24:56.812506  1869 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:24:57.042675  1975 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.003s
I20260709 12:24:57.042874 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:57.043684 30989 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:24:57.043299  1975 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.043782 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:57.044574 30992 raft_consensus.cc:2410] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:24:57.045099  1886 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:57.045444  1975 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:57.045499  1886 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:57.045644 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:57.046554  1886 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:56.916064  2007 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:57.044448 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:57.038546 31136 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:57.048663 31690 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 5: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:24:57.020488  2013 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.407s	user 0.001s	sys 0.000s
I20260709 12:24:57.042639  1869 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:57.059719 30989 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:24:57.060335 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 307438us
I20260709 12:24:57.067063 30985 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:24:56.950454  2018 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:24:56.961596 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:57.047880  1886 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:57.088378 31141 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:56.920471  1920 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:57.024893  1855 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:57.088686 31136 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:56.878830  2005 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:24:57.093248 30987 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:56.922966  2019 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:24:57.109360  2007 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:57.097174 31138 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:24:57.155290 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:57.156176 31138 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:24:57.129733 31033 leader_election.cc:336] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.531s (SENT)
I20260709 12:24:57.132620 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:57.157955 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.159235 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:57.094092 31144 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.160018 31138 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.124104 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:56.812239  1968 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:56.951967  1851 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.000s
W20260709 12:24:57.115969 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 302292us
I20260709 12:24:57.137782 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:57.174577 31141 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:57.087815 31143 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:57.175602 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:57.176128 31143 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:57.176589 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.176280 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 266960us
I20260709 12:24:57.177893 30986 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:57.179394 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:57.179945 31141 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:56.871271  2021 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:57.126741 30989 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:57.141569 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:57.185089 31061 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:57.188627 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.162209 31137 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:24:57.168798 31033 leader_election.cc:336] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.578s (SENT)
I20260709 12:24:57.190826 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.192116  1997 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.192329 31144 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:57.192613  1997 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.195581  1997 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:57.183043 31138 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:57.198382 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:57.199120 31138 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:56.915709  2028 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.224s
I20260709 12:24:57.192894 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:57.201699 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:57.169164  1893 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:57.207767 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:57.209276 30987 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:57.209331 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.178673  1920 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:57.211592 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:24:57.212666 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 1.14222e+06us
I20260709 12:24:57.213624 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:57.218560 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:57.186334  1700 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:24:57.220706 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:57.222244  1493 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:57.222389  2041 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.222802  1493 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:57.223016  2041 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.223803  1493 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.225169  2041 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:57.225464  2020 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:57.226549 31696 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:24:57.228191 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.229213 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:57.229779 31696 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:57.207253  1997 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:57.243427 31061 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:57.183219  1914 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:24:57.207752  1881 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:57.266460  1968 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:57.262912  1758 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.001s
I20260709 12:24:57.266812  1881 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.267248 30989 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
W20260709 12:24:57.270263 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 61022.3us
I20260709 12:24:57.271245 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.246713 31141 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:57.183135  1851 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.244930 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:57.259239  2021 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:57.262897  1923 consensus_meta.cc:287] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.000s
I20260709 12:24:57.263248  2007 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.265007 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 522) took 1020 ms. Trace:
I20260709 12:24:57.265065  1915 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:57.260226  1418 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:57.208122  1973 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:24:57.218830  2002 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:57.275213 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:57.289973 30987 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:57.291131 31143 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:57.298352 31141 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:24:57.297744 31696 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:24:57.300881 31058 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:24:57.303938 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 59101.9us
I20260709 12:24:57.321465  1418 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:57.326349  2038 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.328572  2038 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.328956  1881 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:57.328820  1418 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.330703  2038 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:57.332546  1985 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.263s	user 0.001s	sys 0.002s
W20260709 12:24:57.334133 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 59006.8us
I20260709 12:24:57.334520  1851 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:57.334903 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:57.335989 31144 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:57.268535  1920 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.335999 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.319221 31142 rpcz_store.cc:276] 0709 12:24:56.245052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.245149 (+    97us) service_pool.cc:224] Handling call
0709 12:24:56.245477 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:56.365334 (+119857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.365373 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:56.377860 (+ 12487us) log.cc:844] Serialized 52 byte log entry
0709 12:24:56.601493 (+223633us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:56.601530 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.601594 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.264793 (+663199us) raft_consensus.cc:1685] finished
0709 12:24:57.264815 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.264950 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51545,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13713,"thread_start_us":316219,"threads_started":2,"wal-append.queue_time_us":298379}
I20260709 12:24:57.242543  2020 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.325910 30983 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
I20260709 12:24:57.327386 31143 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:57.329952  1418 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:57.297401 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 82838.7us
I20260709 12:24:57.339147 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:57.337428  1830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.432s	user 0.006s	sys 0.007s
I20260709 12:24:57.297904  2027 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:57.349063 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:57.319825 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 91711.1us
I20260709 12:24:57.326614  1968 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:57.343165 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.239816  2032 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
W20260709 12:24:57.342604 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 127342us
I20260709 12:24:57.344844 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:57.319021  1997 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.342123  2002 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.242974  2041 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:24:57.343938  1474 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:57.348852 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:57.352308  1830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.457s	user 0.007s	sys 0.008s
I20260709 12:24:57.298470  1928 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.369753 31060 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:57.386113 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:57.386416 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:57.386874 31064 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:57.387722 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:57.388249 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:57.346506  1923 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.336488  1950 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.346928  1995 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:57.325745  1915 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.394446 31696 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:57.384629 31061 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: [11.8-13.9]
W20260709 12:24:57.384953 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.385154 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:24:57.373549 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 204779us
W20260709 12:24:57.424955 31033 leader_election.cc:336] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.625s (SENT)
I20260709 12:24:57.425402 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.373201  2025 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:57.382763 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:57.426635 31033 leader_election.cc:336] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.665s (SENT)
I20260709 12:24:57.387006 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:24:57.424883  1950 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.395531 31698 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:57.456528 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.412388 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:57.382670  1973 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:57.420002  1872 consensus_meta.cc:287] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.756s	user 0.004s	sys 0.006s
I20260709 12:24:57.390588  1893 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 12:24:57.392302  1995 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:57.482365 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 100196us
W20260709 12:24:57.486091 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 94197.4us
I20260709 12:24:57.494094 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:57.494589 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:57.494913 31144 raft_consensus.cc:2393] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 5: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:57.495448 31142 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:57.496573 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:57.497299 31142 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:24:57.498294 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:57.420203  1920 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:24:57.511178 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 84537.9us
I20260709 12:24:57.508698 31143 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:57.425150  1968 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:24:57.486796  2044 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:24:57.461409  2053 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.452204  2025 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:57.527437  2025 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:57.528964  1968 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.512255 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:57.377333  1997 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:57.512255 31033 leader_election.cc:336] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.681s (SENT)
I20260709 12:24:57.500834  1995 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:57.422835  1923 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:57.526386 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 139448us
I20260709 12:24:57.526986  2038 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:57.512822  1830 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:57.532006  2057 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:24:57.570147 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 82107.9us
I20260709 12:24:57.575906 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:57.537230 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:57.513995  1969 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.573231 31062 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:24:57.519268  1851 raft_consensus.cc:2764] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:57.513286  2049 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.577584 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:57.494968  1872 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:57.584563 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 98035.2us
I20260709 12:24:57.587612 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:57.571050  1973 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:57.588438 31696 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:24:57.515774  1920 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.571498  2038 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.637532 31689 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:57.536504  2047 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:24:57.578904  1758 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:57.582782  2052 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:57.593679  2038 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:57.658002 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:57.658798 31144 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:57.646220 31062 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:24:57.535849  2053 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.651859 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:57.641525  1358 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:57.582402  1869 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.653522  1915 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:57.654697  1997 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:57.656138  1872 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:24:57.656375  2046 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:24:57.659493  2069 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:24:57.662004 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:57.590404  2021 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:24:57.594519  1851 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:24:57.642596  2049 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:57.595453  1700 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.583088  1969 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.636647  1973 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:58.002154 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 489880us
W20260709 12:24:57.807797 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.103466 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.982580 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:58.104895 30991 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:58.105207 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 453425us
I20260709 12:24:58.105785 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:24:58.108219 30991 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2007) took 1676 ms (client timeout 2024 ms). Trace:
W20260709 12:24:58.108705 30991 rpcz_store.cc:269] 0709 12:24:56.431955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.814035 (+1382080us) service_pool.cc:224] Handling call
0709 12:24:58.108139 (+294104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:57.747582  2064 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:24:58.155452 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:58.025669  1886 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.931s	user 0.011s	sys 0.000s
W20260709 12:24:57.989811  2055 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:24:57.751204  1915 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:57.899669  1920 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.047921  1869 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:24:57.812536  2061 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:24:57.813123  2059 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 12:24:57.813717  2060 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:24:57.807535  2068 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:58.021785  2025 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:57.748132  1893 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.963269  2054 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:58.180647 31141 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:24:58.111732  2041 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.191922 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1782) took 1240 ms. Trace:
I20260709 12:24:58.191618  2041 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.192858  1969 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.192317 31137 rpcz_store.cc:276] 0709 12:24:56.951589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.230102 (+278513us) service_pool.cc:224] Handling call
0709 12:24:57.246172 (+ 16070us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.246547 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.246583 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:57.246994 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 12:24:57.378273 (+131279us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:57.378308 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.378377 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.191734 (+813357us) raft_consensus.cc:1685] finished
0709 12:24:58.191752 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.191868 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":130973,"threads_started":1,"wal-append.queue_time_us":391272}
I20260709 12:24:57.771032  2067 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.111438  1700 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.902271  2053 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:57.899847  1923 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.153105  2078 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:58.194669  2054 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:57.659861  2050 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.532s
I20260709 12:24:57.997895  2069 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.187739  2068 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:58.197391  2067 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:58.204114  2068 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.205482  1509 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.206001  1509 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:58.206076  1700 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.198495  2072 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:24:58.199150  1995 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.625s	user 0.006s	sys 0.006s
I20260709 12:24:58.194890  2025 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:57.901991  2038 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.206844  2054 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.205492  2068 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.221503  2041 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.218902  2067 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.210988  2069 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.275116  2067 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:58.321345 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 213893us
I20260709 12:24:58.323271 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:58.331055 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:58.335850 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2215) took 1432 ms. Trace:
I20260709 12:24:58.335954 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:58.336520 30992 rpcz_store.cc:276] 0709 12:24:56.903743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.046812 (+143069us) service_pool.cc:224] Handling call
0709 12:24:57.047179 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.049334 (+  2155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.049361 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:57.086690 (+ 37329us) log.cc:844] Serialized 26 byte log entry
0709 12:24:57.472223 (+385533us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:57.500439 (+ 28216us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.641225 (+140786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.641293 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.335129 (+693836us) raft_consensus.cc:1685] finished
0709 12:24:58.335218 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.335602 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15905,"prepare.queue_time_us":9590,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":10157,"raft.queue_time_us":29523,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":25207,"thread_start_us":461692,"threads_started":2,"wal-append.queue_time_us":714943}
I20260709 12:24:58.341176 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:58.343268 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:58.344703 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:58.346822 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 244335us
I20260709 12:24:58.175877 31062 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:24:58.356325 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:58.361420 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:58.369603 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.198822 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 514) took 2721 ms. Trace:
I20260709 12:24:58.334156 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:57.659677  1493 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.516s
I20260709 12:24:58.431476  1897 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:58.431722  1509 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.195860  1886 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.434650  1700 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:58.570992  2069 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.482194 31137 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:58.219650  1995 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.505544 31691 rpcz_store.cc:276] 0709 12:24:55.476898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.477008 (+   110us) service_pool.cc:224] Handling call
0709 12:24:55.477300 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.073224 (+595924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.198571 (+2125347us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54004a2c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:58.198765 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1362707712}
I20260709 12:24:58.567370  1509 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.480959  1923 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.576012  2071 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:58.579475 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 245624us
I20260709 12:24:58.580853  2050 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:58.581290  2074 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.582237  2053 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.585909  1889 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.586604  1893 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.587410  1897 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:58.588833 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 227680us
I20260709 12:24:58.483282  1973 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:58.571010  1886 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.599947 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.600083  2038 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.603327 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 817) took 1938 ms. Trace:
W20260709 12:24:58.608054  1994 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 2.349s	user 0.011s	sys 0.016s
I20260709 12:24:58.609616  1995 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:58.610518 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1934) took 1225 ms. Trace:
I20260709 12:24:58.616991 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:58.612628 31141 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:58.615669  2050 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.622841 31693 rpcz_store.cc:276] 0709 12:24:56.664891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.665133 (+   242us) service_pool.cc:224] Handling call
0709 12:24:58.602708 (+1937575us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b540076c354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:58.603008 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1965275520}
I20260709 12:24:58.625478 31064 rpcz_store.cc:276] 0709 12:24:57.385050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.387467 (+  2417us) service_pool.cc:224] Handling call
0709 12:24:57.387805 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:57.469972 (+ 82167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.470010 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:57.470052 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:57.470082 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.610017 (+1139935us) spinlock_profiling.cc:243] Waited 82 ms on lock 0x7b54003971d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:58.610075 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.610340 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188507264}
I20260709 12:24:58.625042  1893 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.630859  1973 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.627310  1358 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:58.634315  1893 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.634908  1358 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.629140  2050 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.637548 31064 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:24:58.622545  1889 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.644963  1889 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:58.647146 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:58.647689 31135 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.649014 30984 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:24:58.651620  1914 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.652043  1914 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:58.652887  1897 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:58.637868  1358 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.655256 31137 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:24:58.658447 31136 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.653913 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:58.662153 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 580) took 1144 ms. Trace:
I20260709 12:24:58.662820 30992 rpcz_store.cc:276] 0709 12:24:57.515347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.364803 (+849456us) service_pool.cc:224] Handling call
0709 12:24:58.365340 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:58.365922 (+   582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.365951 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.585109 (+219158us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.629393 (+ 44284us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:58.629435 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.629495 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.659652 (+ 30157us) raft_consensus.cc:1685] finished
0709 12:24:58.659675 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.659810 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":188003,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":21083,"thread_start_us":43800,"threads_started":1,"wal-append.queue_time_us":71379}
I20260709 12:24:58.658443  1886 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.664628 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:58.666323 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:58.666679 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.667586 31141 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:58.668533  1758 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:58.668874  1973 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.669025  1758 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:58.666339 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:58.671929  1961 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:58.675083 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.675514  1961 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:58.676012 31060 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:58.677489 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.677070  1961 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.675118 31062 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:58.673702  1758 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.679930 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.680085  1758 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:58.678158 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.680902 31060 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:58.682063 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.682325  2053 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:58.683377  2053 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.682747 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.683820  2053 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:58.685037 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.683674 30992 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:58.683640 31060 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:58.686560 31062 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:24:58.683718 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 187031us
W20260709 12:24:58.692348 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 71140.9us
I20260709 12:24:58.693112 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:58.693434 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:58.694540  1964 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:58.695000 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:58.695168  1964 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:58.695988  2074 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.706400 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:58.707238 31060 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.710652 31064 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:24:58.711674 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.653923  1914 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.713610 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.691402  1889 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.717643 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.733898 31137 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 12:24:58.735162 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 46435.1us
I20260709 12:24:58.738786 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.739001 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.739785 31139 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:24:58.739602  1889 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.737048  1973 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:58.742062 31065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 746) took 2748 ms (client timeout 2036 ms). Trace:
I20260709 12:24:58.742249  1973 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:58.742697  1973 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:58.742547 31065 rpcz_store.cc:269] 0709 12:24:55.993899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.026402 (+ 32503us) service_pool.cc:224] Handling call
0709 12:24:58.741641 (+2715239us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54001b9c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:58.741857 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509034240}
I20260709 12:24:58.743929 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 754) took 2139 ms. Trace:
I20260709 12:24:58.744318 31067 rpcz_store.cc:276] 0709 12:24:56.604405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.604521 (+   116us) service_pool.cc:224] Handling call
0709 12:24:56.627409 (+ 22888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.628075 (+   666us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:56.647148 (+ 19073us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.743037 (+2095889us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b18003ec4a0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a
0709 12:24:58.743379 (+   342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:58.743478 (+    99us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:58.743551 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.743640 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.743760 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214,"prepare.queue_time_us":63467,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":2354,"raft.queue_time_us":336,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5516,"spinlock_wait_cycles":70792576,"thread_start_us":106668,"threads_started":1,"wal-append.queue_time_us":446941,"wal-append.run_cpu_time_us":1038,"wal-append.run_wall_time_us":1004376}
I20260709 12:24:58.747970 31060 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:58.746881 31136 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:58.750119 31135 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:24:58.755934  2054 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:58.759424  1951 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:24:58.758632  1964 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.762111 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.764575 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.765388 31137 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:58.766491 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 747) took 2718 ms. Trace:
I20260709 12:24:58.766884 31058 rpcz_store.cc:276] 0709 12:24:56.047962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.170350 (+122388us) service_pool.cc:224] Handling call
0709 12:24:56.170676 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:57.327822 (+1157146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.766048 (+1438226us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001a6dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:24:58.766329 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2598279040}
I20260709 12:24:58.768725  2053 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.773550  2074 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:58.773973  2071 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:24:58.774857 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.775200 31137 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:58.774156 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.776149  2081 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:58.776474 31139 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
W20260709 12:24:58.776998  2068 consensus_meta.cc:287] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.006s
I20260709 12:24:58.777671 30991 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:24:58.762234  2080 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:58.778637 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.778939 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.780395 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 520) took 1583 ms. Trace:
I20260709 12:24:58.780532  2020 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:58.780864 31689 rpcz_store.cc:276] 0709 12:24:57.196961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.197086 (+   125us) service_pool.cc:224] Handling call
0709 12:24:58.779981 (+1582895us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54005e6ad4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:58.780199 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956897024}
I20260709 12:24:58.782545  1835 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:58.783049  1835 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:58.783558  1835 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.785292  2035 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:58.785976  2054 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.786506  1493 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.786799  2000 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.786744  2054 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.787443  2000 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:58.787207  1493 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.788715  2000 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.789769  1493 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.789985  2000 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.792613  1950 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:58.793044 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:58.793197  1950 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:58.793941 31064 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:24:58.794694  1573 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.796133  1573 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.798177 31144 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:58.804410  2050 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.805248 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:58.805837  1923 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.805941 30988 raft_consensus.cc:2376] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5. Re-sending same reply.
W20260709 12:24:58.807090 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.808465  2017 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.809082  2017 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.814467  2074 consensus_queue.cc:1243] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:58.814764  2054 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:58.815531 30955 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.821854 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1772) took 1989 ms. Trace:
I20260709 12:24:58.778663  2080 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:58.299700 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:58.824251 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.826102 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.825026  2080 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.799795  1889 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.831429  1889 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:58.831910  1889 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.830303  2080 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.822377 31136 rpcz_store.cc:276] 0709 12:24:56.814956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.037698 (+222742us) service_pool.cc:224] Handling call
0709 12:24:57.038020 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.657853 (+1619833us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b58007e26d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb7f00aa 00007f2eeb872447 00007f2eeb88afe7 00007f2eeb88c7de 00007f2eeb896b49 00007f2eeb897416 00007f2eeb893d8f 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4
0709 12:24:58.749796 (+ 91943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.804041 (+ 54245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3297193344}
I20260709 12:24:58.836912 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:58.839787 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.827731  2074 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:58.846768 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:58.849210  1835 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.839279  2068 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:58.850453  2068 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.850835  2068 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.854418  1990 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:58.855029  1990 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.855423  1990 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.855517 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1929) took 1471 ms. Trace:
I20260709 12:24:58.855974 31068 rpcz_store.cc:276] 0709 12:24:57.383636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.389532 (+  5896us) service_pool.cc:224] Handling call
0709 12:24:57.424590 (+ 35058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:57.424776 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:57.448761 (+ 23985us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.854803 (+1406042us) spinlock_profiling.cc:243] Waited 7.07 ms on lock 0x7b18003a7c80. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287
0709 12:24:58.854968 (+   165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:58.855029 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:58.855118 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.855216 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.855363 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24846,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":195922,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":60070,"spinlock_wait_cycles":16272128,"thread_start_us":272564,"threads_started":2,"wal-append.queue_time_us":203726}
W20260709 12:24:58.858671 31690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 812) took 2214 ms (client timeout 2440 ms). Trace:
I20260709 12:24:58.858920  2081 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:58.859100 31690 rpcz_store.cc:269] 0709 12:24:56.644286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.657580 (+ 13294us) service_pool.cc:224] Handling call
0709 12:24:58.858283 (+2200703us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b5400514d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:58.858479 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896447232}
I20260709 12:24:58.859730  2081 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.861548  2053 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:58.844839 31135 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:24:58.866578 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 86029.1us
W20260709 12:24:58.873752 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.882642 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.888293 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.889590  1835 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:58.889992 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:58.890928  1835 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:58.891590  1835 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:58.846019  2074 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:58.870692  2053 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:58.894786 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:58.895073  2054 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:58.892808  1835 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.896627  1889 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:58.898475 30991 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2015) took 1513 ms (client timeout 1623 ms). Trace:
W20260709 12:24:58.924767 30991 rpcz_store.cc:269] 0709 12:24:57.385127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.109888 (+724761us) service_pool.cc:224] Handling call
0709 12:24:58.898188 (+788300us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54000230d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:58.898366 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1012456320}
I20260709 12:24:58.893868  2081 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:58.903337  1965 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:58.925947  1965 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:24:58.927350  1965 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:58.929265  1973 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:58.934247 30990 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2008) took 2501 ms (client timeout 2077 ms). Trace:
I20260709 12:24:58.905000 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:58.886590 31139 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
W20260709 12:24:58.950171 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.921162  2074 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:58.921716  2054 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.922281  2053 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:58.923538 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 574) took 2241 ms. Trace:
I20260709 12:24:59.027577  2071 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.037782  1358 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.040688  2029 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:59.041577  2053 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.035089 30985 rpcz_store.cc:276] 0709 12:24:56.681767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.863336 (+181569us) service_pool.cc:224] Handling call
0709 12:24:57.066652 (+203316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:57.085216 (+ 18564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.923154 (+1837938us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400226850. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:58.923354 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466717568}
I20260709 12:24:59.035109  2054 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.042807 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1777) took 2183 ms. Trace:
I20260709 12:24:59.064811 31140 rpcz_store.cc:276] 0709 12:24:56.858987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.087586 (+228599us) service_pool.cc:224] Handling call
0709 12:24:57.087913 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:57.088179 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:57.093374 (+  5195us) log.cc:844] Serialized 14 byte log entry
0709 12:24:58.619023 (+1525649us) spinlock_profiling.cc:243] Waited 979 us on lock 0x7b18000f2080. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:24:59.042148 (+423125us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:59.042261 (+   113us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:59.042340 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.042439 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.042628 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":304542,"prepare.queue_time_us":4967,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":306355,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":113,"spinlock_wait_cycles":2252032,"thread_start_us":102453,"threads_started":1,"wal-append.queue_time_us":1692042}
I20260709 12:24:59.068533  2029 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.127594  1886 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:59.078346  1358 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.128494  1886 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.129725 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:24:59.131399 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 93210.6us
I20260709 12:24:59.155051 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 742) took 3191 ms. Trace:
I20260709 12:24:59.155666 31063 rpcz_store.cc:276] 0709 12:24:55.962980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.131375 (+168395us) service_pool.cc:224] Handling call
0709 12:24:56.131664 (+   289us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:56.139946 (+  8282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.140001 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:56.140680 (+   679us) log.cc:844] Serialized 52 byte log entry
0709 12:24:58.744469 (+2603789us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b18003ec560. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:24:58.845790 (+101321us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:58.845842 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.867704 (+ 21862us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.154476 (+286772us) raft_consensus.cc:1685] finished
0709 12:24:59.154504 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.154836 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34839,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":13055,"spinlock_wait_cycles":15600000,"thread_start_us":84630,"threads_started":1,"wal-append.queue_time_us":2998044}
W20260709 12:24:59.159180 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 260129us
W20260709 12:24:58.934676 30990 rpcz_store.cc:269] 0709 12:24:56.432171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.933863 (+2501692us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:58.934028 (+   165us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:24:59.175684 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.177189 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.159207  2074 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.163736  1973 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.186605 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:59.172947  2000 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:59.185421 31664 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.848s (SENT) [suppressed 25 similar messages]
I20260709 12:24:59.190296 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.190871 31062 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.191174 30984 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:59.186393 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.197041  1418 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:59.197778  1418 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.195369 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.197955  1995 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:59.200562 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.202541 31060 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.182614  1573 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.193192  1889 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.198915  1965 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:59.208441  1493 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:24:59.186046  1893 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:24:59.219707  1893 consensus_queue.cc:1243] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:59.205181  1573 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.218490  1990 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.219239 31689 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:59.219914 31690 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:59.225636 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:59.231516 31698 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:24:59.231925 31698 pending_rounds.cc:85] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 5
I20260709 12:24:59.232100  1418 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.232164 31698 pending_rounds.cc:107] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 5.6
W20260709 12:24:59.233695 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 61539us
I20260709 12:24:59.233412 31698 raft_consensus.cc:2889] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:59.249637 31690 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:59.251816  2053 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.251492  2066 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:59.291414  2066 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.292379  1573 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.293723  2066 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.295745  2053 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.296598 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.303289 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.304692 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.305948 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.306828 31058 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.309188  2020 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.309414 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:59.309756  2020 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.311456 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:59.315223  2068 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:59.320436 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 104257us
I20260709 12:24:59.320554 31058 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:59.322078 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.323719  1979 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:59.323556  2066 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:59.323608 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:59.326293 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:59.327313  2014 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:59.327975  2014 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:59.328513  2014 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.249114 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:59.280213 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.330631 30990 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
W20260709 12:24:59.331286 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.332671  1995 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.333266  1995 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.335309  1995 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:59.335765  1995 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.336268  1995 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.338300  2066 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.338481  1474 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.339046  1474 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.339389 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:59.252010  2076 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:59.341064  1474 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:59.283100 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:59.284955  1418 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:59.285871 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 519) took 2089 ms. Trace:
I20260709 12:24:59.342393  1951 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:59.342777  1951 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:59.342816 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:59.343448 31693 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.343878  1951 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.345134  1951 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.342016  1418 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:59.339053 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:59.347021  1418 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.341579  1474 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.347584 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:59.347849  1474 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.349247  1418 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:24:59.349438  2091 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:59.282573 31063 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:59.350183 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:59.351280 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.341799 31695 rpcz_store.cc:276] 0709 12:24:57.196640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.197869 (+  1229us) service_pool.cc:224] Handling call
0709 12:24:57.227459 (+ 29590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.227780 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.227817 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.832633 (+1604816us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b2c000b6068. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:24:58.833133 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.888301 (+ 55168us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:58.888350 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.134289 (+245939us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.285479 (+151190us) raft_consensus.cc:1685] finished
0709 12:24:59.285498 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.285693 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6064,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3375,"spinlock_wait_cycles":28354176,"thread_start_us":54764,"threads_started":1,"wal-append.queue_time_us":395457}
I20260709 12:24:59.290704  1990 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.355518  2071 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:59.327351 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 111654us
I20260709 12:24:59.358523  1995 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.350054  1474 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.361013 31693 raft_consensus.cc:2410] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:59.366796  2020 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.370193  2054 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:59.373528 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.374569 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:59.375025 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:59.406445 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49203.1us
I20260709 12:24:59.409823 30990 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:24:59.410959 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.406193 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.376561 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:59.377521 30991 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:59.382200  2014 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.397889  1914 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:59.400861 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 96777.1us
I20260709 12:24:59.417028 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:59.420192 31058 raft_consensus.cc:2410] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:24:59.420672  1851 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:59.421041  1851 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:59.418612  1961 raft_consensus.cc:2764] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:59.427863 31065 raft_consensus.cc:2410] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:59.375613 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:59.429128 30992 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:59.429257 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 183727us
I20260709 12:24:59.413185 31060 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:24:59.400347  2017 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:24:59.432278 31064 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:59.432494 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:59.414654 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.434942 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:59.435423 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:59.422559  1950 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:59.436331  1950 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:59.436667  1758 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:59.437229  1758 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:59.437282  1950 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.438392  1493 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:59.438189  1758 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.439417  1493 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.440016 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:59.441577  1493 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.441895 30987 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 11. Ops: []
I20260709 12:24:59.442528 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 576) took 2265 ms. Trace:
I20260709 12:24:59.442899 30989 rpcz_store.cc:276] 0709 12:24:57.177262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.196844 (+ 19582us) service_pool.cc:224] Handling call
0709 12:24:59.442187 (+2245343us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b5400224a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:59.442348 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110279552}
I20260709 12:24:59.444162 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2224) took 2072 ms. Trace:
I20260709 12:24:59.444056 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.444556 30987 rpcz_store.cc:276] 0709 12:24:57.371493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.374335 (+  2842us) service_pool.cc:224] Handling call
0709 12:24:59.441578 (+2067243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.442320 (+   742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.443971 (+  1651us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54000249d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:59.444110 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4753216896}
I20260709 12:24:59.445374 31060 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:59.446681 31062 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
W20260709 12:24:59.441823 30983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 1998) took 3066 ms (client timeout 1681 ms). Trace:
I20260709 12:24:59.422736  1573 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.427583  1973 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:59.410560  1990 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:24:59.450263 30983 rpcz_store.cc:269] 0709 12:24:56.374933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.375106 (+   173us) service_pool.cc:224] Handling call
0709 12:24:59.441352 (+3066246us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54000249d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:59.441609 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144429440}
I20260709 12:24:59.460312 31139 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:59.430461  2066 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.429924  1995 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:59.468513 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:59.469291 31136 raft_consensus.cc:2376] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7. Re-sending same reply.
I20260709 12:24:59.470400 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:59.478449  1531 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:59.479159  1531 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.482308  1531 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.462879  1474 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.494952  1474 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.496465  2029 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.497313 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 540) took 1905 ms. Trace:
I20260709 12:24:59.497244  1474 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:59.497730 31142 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:24:59.498525  1829 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:59.497783 31143 rpcz_store.cc:276] 0709 12:24:57.591492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.591677 (+   185us) service_pool.cc:224] Handling call
0709 12:24:57.592092 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.038700 (+1446608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.038759 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.199713 (+160954us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.401691 (+201978us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:59.401746 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.401819 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.497002 (+ 95183us) raft_consensus.cc:1685] finished
0709 12:24:59.497022 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.497165 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1445916,"raft.queue_time_us":1447484,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":201,"thread_start_us":179008,"threads_started":1,"wal-append.queue_time_us":258487}
I20260709 12:24:59.499279  1829 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:59.500437  1829 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.505129 31140 raft_consensus.cc:2376] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5. Re-sending same reply.
I20260709 12:24:59.505417 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:24:59.506099 31059 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:59.508628  1886 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:59.511878  1995 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.514935  2014 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:59.516567 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:59.517546  2074 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:24:59.518146 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:59.523134 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:24:59.524636 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 92393.4us
I20260709 12:24:59.524091 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:59.524443 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:59.523878 31135 raft_consensus.cc:2376] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10. Re-sending same reply.
W20260709 12:24:59.553651 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 116951us
W20260709 12:24:59.555322 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49066.2us
W20260709 12:24:59.559547 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.558135  1573 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:59.561971 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:59.562465  1573 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:59.562830 31136 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:24:59.563431  1573 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:59.565505 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 101983us
I20260709 12:24:59.566360 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:59.526687  2098 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:24:59.466928  1973 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:24:59.522904  2080 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:59.525576  2068 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.529989  2074 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:59.580626 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 103976us
I20260709 12:24:59.598539 31139 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.558373  1995 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.603381 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:59.579841  1493 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.609157 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:24:59.616184 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 98155.8us
I20260709 12:24:59.616806 31136 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:59.614058 30992 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:24:59.617762 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:59.581269  2080 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:59.615240 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:59.585026  2053 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.629216 31135 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:59.582655  1886 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:59.610391  1990 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:59.633225 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 66981.5us
I20260709 12:24:59.639365 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:59.643735 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:59.607774  2071 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:59.629048  2080 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.655579 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:59.656375 31142 raft_consensus.cc:2410] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:24:59.656598 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 50693.6us
I20260709 12:24:59.638900  1758 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.640673  1531 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:59.641851  1990 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:24:59.673911 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:24:59.668078  2012 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:24:59.658423  2014 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.676568 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:24:59.673218 31106 connection.cc:256] RPC call timeout handler was delayed by -1.0768811349998941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
W20260709 12:24:59.686372 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.644220  2074 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.659561  2066 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.650701  1493 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.701865 31136 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:24:59.704768 31063 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:59.704802 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:24:59.701936 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 58329.6us
I20260709 12:24:59.674971  2054 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.676888  1990 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.652381  2053 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.704448  1951 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.709348 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.711931  1886 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:59.712560  1531 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:59.657549  1973 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.687785  2050 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.741058  1951 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.744717 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.745677 31061 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:24:59.747009 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.748816  2047 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.749464  2047 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:24:59.751055 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 83318.5us
I20260709 12:24:59.751726  2012 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:59.752169  2012 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:59.752619  2012 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:59.753283  2103 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:59.751909 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:59.756341 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:59.756724 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:24:59.758148 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:59.758533  1418 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:24:59.759042 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.759532 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.760624 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:59.760301  1851 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.760980  1418 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:59.761404 31140 raft_consensus.cc:2410] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:24:59.730118  1950 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:24:59.762341 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:24:59.762863  1950 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:59.763231  1950 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:24:59.764058  1950 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:59.766536 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.773182 31033 connection.cc:256] RPC call timeout handler was delayed by -1.626530940000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 12:24:59.740914 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.775352 31143 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:24:59.738639  2102 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.140s	user 0.000s	sys 0.006s
I20260709 12:24:59.761584  1418 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.704758  2014 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.696971  1758 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:59.784251 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.782554 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:59.811900 30989 raft_consensus.cc:2410] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:24:59.806187 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:59.813138 30984 raft_consensus.cc:2410] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:24:59.732858  1474 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:59.808732 31033 connection.cc:256] RPC call timeout handler was delayed by -1.1774088209999718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 12:24:59.780157 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:59.758546  2025 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:59.807224 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:24:59.761423  1851 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:59.814828 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:59.811724 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.775616  1951 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:24:59.809361 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:24:59.780563  1965 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:59.814863 30983 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:24:59.823968 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:24:59.819864 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.821377 31135 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
W20260709 12:24:59.827003 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.809547  2054 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:24:59.817143  1923 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.816241  1474 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:24:59.834209 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.811491  2069 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.836375 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:24:59.841212  2110 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:24:59.842422  1418 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.847759 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.848904 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.849332  1758 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.836743  2093 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:59.850478  2093 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:24:59.850899  2093 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.851190 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.852018 31062 raft_consensus.cc:2410] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:24:59.851832  1923 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.852766 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.853691 30988 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:24:59.853859 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:59.854425 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.854641 31033 leader_election.cc:336] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:24:59.854727  2069 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.855904 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.870857 31139 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:24:59.871145 31060 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.857321  2053 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.836496  2054 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.859021 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:59.859730 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:24:59.859903  2074 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.828088  1893 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:59.860829  2072 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:24:59.891613 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 108601us
I20260709 12:24:59.860538  1531 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.862349 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:24:59.850833  1474 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.849918 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:24:59.853952  1923 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.876740  1493 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.876740 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:24:59.877650 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 98467.2us
I20260709 12:24:59.835111  2038 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:24:59.883978 31693 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:24:59.893605 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:59.857605  2012 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.898898 31062 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:24:59.940515 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.941051 31062 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:59.941138 31063 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:24:59.941541  2012 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.830384  2080 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:24:59.907735 31068 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:24:59.943075 31063 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:59.936161 31060 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:24:59.898665  1474 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.945453  1886 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.903867  1358 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.937873  2054 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.915071  2014 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.912099 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:24:59.898419  1893 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:24:59.949539 31106 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:24:59.945931  1886 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:24:59.956488 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 101914us
I20260709 12:24:59.964184 31058 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:24:59.967124  1877 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:24:59.967811  1965 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:59.968204  1965 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.968317  1886 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:24:59.964547 31058 pending_rounds.cc:85] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 3
I20260709 12:24:59.969175 31058 pending_rounds.cc:107] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:24:59.968619 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.969545 31058 raft_consensus.cc:2889] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:59.970206 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 589) took 1074 ms. Trace:
I20260709 12:24:59.968598  1965 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.968873  2000 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:59.970686 30988 rpcz_store.cc:276] 0709 12:24:58.890864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.894509 (+  3645us) service_pool.cc:224] Handling call
0709 12:24:59.959433 (+1064924us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b540020f154. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:24:59.964889 (+  5456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2149247872}
I20260709 12:24:59.953075  2038 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.985525 31060 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:24:59.986912  1418 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.988960  1893 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:24:59.974047  2000 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:59.965727  2012 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:59.990486  2014 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:24:59.990546  1877 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:00.014149  1877 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:00.014420  2080 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:00.015372  1877 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.016934  1877 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.992369 31058 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:24:59.990134 31693 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:59.982586 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:24:59.898195  2093 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.008867 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 772) took 1208 ms. Trace:
I20260709 12:24:59.995960 31068 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:00.019268 31067 rpcz_store.cc:276] 0709 12:24:58.800098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.532461 (+732363us) service_pool.cc:224] Handling call
0709 12:24:59.532878 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.533332 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.533373 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.534043 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.818606 (+284563us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:59.818678 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.818771 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.008666 (+189895us) raft_consensus.cc:1685] finished
0709 12:25:00.008688 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.008808 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1773,"raft.queue_time_us":20945,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":19279,"thread_start_us":284088,"threads_started":1,"wal-append.queue_time_us":327000}
W20260709 12:24:59.997501 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 85470.7us
I20260709 12:25:00.020913  1418 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.021540 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:00.001291 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.022449 31139 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:00.023097  1418 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.023217 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:00.024667 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:00.004609 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:00.011365 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:00.008268  1493 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:25:00.006892  2000 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:00.013341  2122 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:00.012990  2014 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.025694 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:00.030169  1995 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:00.038573  1995 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:00.038777  2124 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:25:00.039790  2014 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.039561  1995 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.040488  1951 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:00.040903  1951 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.041236  1951 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:00.030882 31064 raft_consensus.cc:2410] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:25:00.043495 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:00.044792 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:00.045773 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:24:59.994329  2038 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.046447  1829 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:00.036823  2074 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:00.047216  1829 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:00.048125  2038 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:00.048621 30988 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:00.050511  2080 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:00.050637  1851 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:00.051486 30988 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:25:00.051973  2080 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:00.052337  2080 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:00.053249  2080 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.055562 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:00.056422 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.056483 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.057003 30983 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:00.057132 30986 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:00.057464  1829 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.058441 30987 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:25:00.058072  1829 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.058776 30987 pending_rounds.cc:85] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 8
I20260709 12:25:00.059068 30987 pending_rounds.cc:107] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:25:00.059295 30987 raft_consensus.cc:2889] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:00.060384  1829 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.062172  2025 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:00.062614 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:00.062985  2025 consensus_queue.cc:1243] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:00.063030  1474 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:00.063328  1877 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:00.064194  1877 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:00.064426  2027 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:00.064587  1877 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.064916  2027 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:00.065014  1877 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.066159  2027 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.067343  2027 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.070765  2014 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.070890  1829 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:00.071300  2014 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.072412 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.073257  2014 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.073711  2025 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:00.074074  2025 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:00.075079  2025 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.057201 30988 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:25:00.076683  1923 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:25:00.031543 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 78322.5us
I20260709 12:25:00.022470  2012 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:00.076241  2069 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:00.034943  1418 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:00.035187 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:00.092115  1923 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:00.061965  2047 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:00.093608 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1787) took 1361 ms. Trace:
I20260709 12:25:00.093993 31141 rpcz_store.cc:276] 0709 12:24:58.732208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.737142 (+  4934us) service_pool.cc:224] Handling call
0709 12:24:58.740990 (+  3848us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.043611 (+302621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.043654 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.184926 (+141272us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.427097 (+242171us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:59.612377 (+185280us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.613504 (+  1127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.613599 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.091555 (+477956us) spinlock_profiling.cc:243] Waited 76.3 ms on lock 0x7b18003b0500. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:00.093341 (+  1786us) raft_consensus.cc:1685] finished
0709 12:25:00.093435 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.093554 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":309481,"prepare.queue_time_us":23953,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":303423,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":4820,"spinlock_wait_cycles":175552896,"thread_start_us":393558,"threads_started":2,"wal-append.queue_time_us":271475}
I20260709 12:25:00.095206  1951 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:00.097049  2126 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
I20260709 12:25:00.097695  1951 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:00.024693  1990 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.092991  1923 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:00.098456  1923 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:00.099555  1990 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:00.100296 31690 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:00.099570  1923 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.089386 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:00.117967 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:00.101143 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55437.7us
I20260709 12:25:00.123284 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:00.125823 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.125900 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:00.126569 30983 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:00.090605 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:00.139590 30990 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:00.140468 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.141587 31060 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:00.144387 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.145187 31141 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:00.148296 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:00.149576 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 278798us
W20260709 12:25:00.151161 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 64804.9us
I20260709 12:25:00.151556 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:00.115690  2080 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.125706  2025 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:00.101440  2047 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:00.133714  2012 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:00.134316  1964 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:00.067396  1965 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.090582  2014 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:25:00.144558  2050 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:00.166260 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:00.133890  1418 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:25:00.160990  1474 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:00.175305 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:00.181926  1889 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:00.182358  2038 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.182710  1889 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.184804  1889 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.176419  2050 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:00.188069  2050 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:00.189683  2038 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:00.166709 31661 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:00.177752 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.178084  1877 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.191277  2000 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.192148  1965 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:00.193260  2000 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.193416 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.194307 31067 raft_consensus.cc:2410] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:00.197027 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:00.197213 30983 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:00.158108  1923 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.174480  1886 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:00.166676  2012 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.165035  2047 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.203823 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:00.165668  1990 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:00.174005  1964 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.201119 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:00.200280 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:00.182001  2131 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:25:00.206570 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:00.204387 30986 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.204954 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:00.205288 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:00.211341 31063 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.216512 31059 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:00.157603  1961 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:25:00.193269  1877 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.214980 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:00.210388  2047 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.210525  1889 raft_consensus.cc:2764] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:00.226599  2000 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:00.166960  2066 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:25:00.212919  2017 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:00.230988 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 65897.6us
I20260709 12:25:00.232283 30986 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:00.204008  1493 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:00.204541  1886 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.246863  1886 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.248267 31141 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:00.203750  2012 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:00.212147  1964 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.229230 31063 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:00.227013  2014 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:00.258919  2014 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:00.259827  2014 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.241897  2017 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.246405  1493 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:00.262725  2017 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:00.263447  1493 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:00.246099 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.202416  2029 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:25:00.312873 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 69406.8us
I20260709 12:25:00.316438 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:00.328265 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 113385us
I20260709 12:25:00.324631  2025 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.315025  2038 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:00.343093 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.368527 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.346465 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.347523 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:00.318698  1851 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.266263  2069 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:00.328837  1877 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.391470 31693 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:25:00.392409 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 189324us
I20260709 12:25:00.368113  2038 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.375185  2072 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.410210 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:00.414562 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:00.422863 31135 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:00.425213 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:00.415234 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:00.391065  1979 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:00.426539 31143 raft_consensus.cc:2376] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8. Re-sending same reply.
I20260709 12:25:00.417485 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:00.418433 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:00.418671 31692 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:00.419269 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:00.419936 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:00.320736  2054 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:25:00.434175 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:00.420784 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.383989  2070 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:00.416791 31693 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:25:00.428776 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:00.390844  1509 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.432310 31689 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:00.435343 31141 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:00.433842 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.441279 31140 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:00.328467  1990 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:00.449966 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.439316  1474 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.450178  1990 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.451123 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.450958  1474 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.451690 31136 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:25:00.452559 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.453287 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.453651  1474 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:00.448262 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.465893 31137 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:00.467160 31137 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:00.467860 31144 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:00.468305 31137 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:00.449012 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.470770 31143 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:00.449159 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:00.454690  2147 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:00.455089 31138 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:00.455878  2149 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:00.456159  2137 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:25:00.457677  2129 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:25:00.376449  2025 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.475481 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.446971  1509 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.453266 31136 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:00.420924  2038 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.427641  1965 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:00.386456  1851 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.490455 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.525976 30992 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:00.494621 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.526674 30987 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:00.499100 31135 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:25:00.496238 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 69203.9us
I20260709 12:25:00.439458  2068 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:00.497030 30983 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:25:00.528564 30983 pending_rounds.cc:85] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 5
I20260709 12:25:00.528929 30983 pending_rounds.cc:107] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:25:00.529194 30983 raft_consensus.cc:2889] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:00.413312  2066 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:00.430696  2072 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.536048 31139 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:00.533699 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 98499.5us
I20260709 12:25:00.521373 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.495941  2080 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:00.441767  1877 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.535698  2066 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:00.496686  1758 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.562112 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.566489 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.567271 30992 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:00.564908 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.565466 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.571149 31138 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:00.572451 31143 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:00.526072  2093 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:00.574124 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.574970 31136 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:00.576306 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.574321 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:00.580072 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:00.592420 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 263455us
I20260709 12:25:00.622570 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:00.490581  2149 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.624789 30991 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:00.558101  1965 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:00.617300 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:00.634896 31139 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:25:00.679991 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 105753us
I20260709 12:25:00.563802  2148 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:00.683959 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.621765  1474 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.688315  2012 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.691365 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2233) took 1314 ms. Trace:
I20260709 12:25:00.728480 30990 rpcz_store.cc:276] 0709 12:24:59.377168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.433694 (+ 56526us) service_pool.cc:224] Handling call
0709 12:24:59.442799 (+  9105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.139992 (+697193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.691136 (+551144us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54003453d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:00.691309 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1602067840}
I20260709 12:25:00.575711  2146 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.694783 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.696947 31140 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:00.570421  2027 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:00.698994 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:00.699803 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.730780 31063 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:00.701606 31067 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:25:00.707327 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.630914 31138 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:00.704528 31106 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:00.716913 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.733700 31058 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:00.723268 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.575522  1758 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.625906  2074 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:00.735726 31058 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:00.737489  1474 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.723208  2093 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.716269  1418 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:25:00.737646  2146 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.767072 31067 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:00.738642 31059 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:00.738623 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.738961 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:00.739323 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:00.740302 31142 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:00.739707  2027 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.715935  2068 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:00.741231  2012 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.741992  2130 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:25:00.747272 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.574150  2080 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.699465  2054 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:00.706125  2138 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:00.761713 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.763240 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.848521 30988 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
W20260709 12:25:00.764716 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.680663  2029 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:25:00.765499 31138 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:00.737993  1509 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:00.769428 31065 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:25:00.723883  2069 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:00.847852  2138 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:00.854624  2069 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.855103  2138 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.805277  1951 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
W20260709 12:25:00.754125  2144 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.063s	user 0.000s	sys 0.006s
I20260709 12:25:00.856968  2069 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:00.761137  2142 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:25:00.777181 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:00.779599 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.793030 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:00.806277 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.858234 30989 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:00.750240  1829 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:00.824550  2050 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.636s	user 0.006s	sys 0.006s
I20260709 12:25:00.825695  1877 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.765743  2074 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.827040  2145 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:00.827293  1474 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.827682  2025 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:00.828655  1758 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:00.830086  1886 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.583s	user 0.007s	sys 0.001s
W20260709 12:25:00.831729  1964 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.000s
I20260709 12:25:00.781348  2066 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.836800  2139 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.802431  2068 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:00.841382 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.004s
I20260709 12:25:00.819939 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.853084 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:00.853216  2054 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:00.935189 30988 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:00.939692 30992 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:00.856833  1968 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.159s
I20260709 12:25:00.801187 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:00.815850 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.764212  2148 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:00.765832  2093 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:00.826534  2072 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.867959 31064 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:25:00.872329 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 95244.7us
I20260709 12:25:00.876583 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:00.944746 30989 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.856380  2138 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:00.940531 30990 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:00.940793 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.008s	sys 0.005s
I20260709 12:25:00.941035 31059 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
W20260709 12:25:00.949028 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 682115us
W20260709 12:25:00.936592 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 190112us
W20260709 12:25:00.953508 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.865316  1950 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:00.959476 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.964294 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.964960 31059 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:00.965051 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:00.965595 31064 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:00.966509 31689 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:00.967221 31141 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:00.968326 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:00.968988 31141 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:00.972401  2074 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:00.974897 30992 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:25:00.975859 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:00.976080 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.976490 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.871183  2050 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.978765 30987 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:00.979422 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:00.980018 30985 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:00.981557 30989 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
W20260709 12:25:00.984609 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.949813  1886 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.989590 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:00.907619  1851 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.036000 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:01.036597 30983 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:01.042331 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:00.929287  2150 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.907274  2029 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:00.982959 30991 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
W20260709 12:25:01.044152 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54626.5us
I20260709 12:25:01.057294 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:01.038929 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 78753.5us
I20260709 12:25:01.057907 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:01.065874 30985 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:01.058944 30992 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:01.061786 31689 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:25:01.066334 31144 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:01.061646 30990 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:25:01.067377 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:01.068488 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.035871  2066 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:00.943729  1964 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:00.990049  1886 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.071461 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:01.074491 31067 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:01.073478 31141 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:00.862075  1965 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.078044  1889 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:01.073249  2150 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.068329  2029 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.080050  2068 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.080158  2029 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:01.081382  2093 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.081985  2027 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.076140  2074 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:01.082705  2154 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:25:01.082873  2074 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.084496 31062 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:25:00.951407  2139 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.100363 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:01.076123  2050 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.070859  1968 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:00.908332  2145 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.041787  2143 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:25:01.070715  1923 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:25:01.082623  2054 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.110191 31144 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:25:01.162302 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 120145us
W20260709 12:25:01.360450 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 212956us
I20260709 12:25:01.498750 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:01.507748 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:01.393924 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.370196 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.302591 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:01.501607 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:01.109321 31058 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:01.126631 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:01.509940 31063 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:01.374135 31139 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:25:01.287256 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 219942us
I20260709 12:25:01.512671 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:01.525600 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.071175  2038 consensus_meta.cc:287] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.543s	user 0.003s	sys 0.013s
W20260709 12:25:01.197841  2126 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:01.542307 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:01.391891  2047 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.150s	user 0.014s	sys 0.000s
I20260709 12:25:01.551858 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:01.543241  2069 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.552173  2047 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.552973  1851 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:01.391283  2139 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.554440  2047 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.552116 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:01.198207  2017 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.935s	user 0.010s	sys 0.003s
I20260709 12:25:01.349336  2146 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.512030  1968 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.556103  1968 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:01.545861  2172 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 12:25:01.543079  2157 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.558599  2069 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:01.227366  2162 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:01.549153 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:01.125303  2050 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.555243  2017 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.558442 30986 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 12:25:01.327903  2171 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:25:01.172287  2074 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.428399  2130 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:01.514889 31064 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:25:01.433404  2164 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.365s	user 0.000s	sys 0.000s
I20260709 12:25:01.188854  2080 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.560688 30955 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:01.105079  1889 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:01.492156  1964 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.559036  2069 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.269861  2117 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:01.566526  2047 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:01.566821 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:01.499557  2076 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.483566  2147 raft_consensus.cc:476] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Not starting pre-election -- already a leader
I20260709 12:25:01.437762  2150 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.101406  2149 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.615921  1886 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:01.616315  1886 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:01.559695  2014 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.001s	sys 0.031s
W20260709 12:25:01.619498  2014 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.219s	user 0.001s	sys 0.031s
I20260709 12:25:01.572537 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:01.583050 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:01.515902  1358 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:01.601720 31064 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:01.585054  2069 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:01.573678  1758 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:25:01.605301 31062 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:01.622820 31064 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:01.608021  2167 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.551s	user 0.000s	sys 0.000s
W20260709 12:25:01.608805 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59723.5us
I20260709 12:25:01.630170 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:01.559839  2068 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:25:01.612545 31696 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:01.632113 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:01.632589  2068 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:01.632776 31068 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:01.633083  2068 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:01.572419  2130 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:01.550858  2038 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.548874  2159 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:25:01.601660  2117 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:01.615603  2076 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.620302 31063 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:01.545159  2072 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:25:01.580865  2080 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:01.574620  2175 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:01.567435  2050 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.629204 31064 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:01.654579 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 77323.5us
W20260709 12:25:01.722865 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 147589us
I20260709 12:25:01.723235  2190 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.723836 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:01.723943  2190 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:01.716481  1950 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.733s	user 0.012s	sys 0.000s
I20260709 12:25:01.142091  2148 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.738318  2017 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:25:01.718818  1990 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:01.738775 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:01.739198  1990 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.627110  1358 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.627818  2178 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.715062  2150 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.728389 31064 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:01.703328 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 73242.8us
I20260709 12:25:01.739549  2017 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:01.706952 31068 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:01.705303  2141 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:01.742192 31061 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:01.744079 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:01.716791  2117 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:01.719159  2080 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:01.746523 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:01.745167  2071 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.759394 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:01.760516 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 198417us
I20260709 12:25:01.761230 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:01.720880  1851 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:25:01.673506  1889 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.642887  2068 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.727325  2146 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:25:01.741328  1990 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.743520  2017 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.743702  1358 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:01.744078  2141 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:01.788435 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 45660.1us
I20260709 12:25:01.762374  2080 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:01.779732  2130 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:01.805377 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:01.768527  2131 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.770550  2071 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:01.818804 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 93886us
I20260709 12:25:01.768676  2076 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.770512  2150 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:01.827775 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.823764 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:01.785048  2017 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:01.824828  2068 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:01.825282  2069 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.826439  2179 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:25:01.798390  2190 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.818804  2145 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.821763  2146 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:01.745576  2178 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.827158  1965 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:25:01.852062  2050 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:01.852681  2069 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:02.101722  2130 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:01.968609 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.064335 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.912413 31061 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2265) took 2224 ms (client timeout 1900 ms). Trace:
W20260709 12:25:02.083279 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.055384 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.888954 31064 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:25:01.852625  2050 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.836263  2076 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:01.838032  2180 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.141s	user 0.000s	sys 0.003s
I20260709 12:25:02.105078 31064 pending_rounds.cc:85] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 6
W20260709 12:25:02.105587 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 300685us
I20260709 12:25:01.836622  2153 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:01.901046  2191 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 12:25:02.118633 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 298370us
W20260709 12:25:02.103320 31061 rpcz_store.cc:269] 0709 12:24:59.687548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.808040 (+120492us) service_pool.cc:224] Handling call
0709 12:25:01.912231 (+2104191us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b54001a72d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:01.912341 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4406360320}
W20260709 12:25:02.022550  2184 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 12:25:01.984805  2029 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.904s	user 0.011s	sys 0.005s
I20260709 12:25:01.873942  1851 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.135300 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:02.022351  2183 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:25:02.132288 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:02.128901 31064 pending_rounds.cc:107] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:25:02.138826 31064 raft_consensus.cc:2889] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:01.854323  2187 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:01.969199  2138 consensus_meta.cc:287] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.005s	user 0.010s	sys 0.005s
W20260709 12:25:01.853246  2185 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:25:02.135877 31065 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:02.110157  2186 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.273s	user 0.001s	sys 0.001s
W20260709 12:25:01.953672  2066 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.880s	user 0.007s	sys 0.007s
I20260709 12:25:02.136785 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:02.021704  2178 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:01.863364  2146 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:02.146456 31064 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 12:25:02.152356 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 391178us
I20260709 12:25:02.128293  2153 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:02.147828 31067 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:02.119406  2069 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:01.833191  1990 consensus_queue.cc:1175] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" responder_term: 8 status { last_received { term: 8 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 8 index: 8 } }
I20260709 12:25:02.133574  2174 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:01.984926  1968 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.154281 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 558) took 1620 ms. Trace:
W20260709 12:25:02.156165 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 1.36311e+06us
I20260709 12:25:02.143647  2187 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.166667 30992 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:02.151537  2175 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:02.168128  2175 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.159724  2138 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.170611  2138 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:02.171032  2138 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.170686  2153 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.171765  2153 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:02.028846  2117 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:02.158560  2177 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.288s	user 0.001s	sys 0.003s
I20260709 12:25:02.138062  2029 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.175559 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:02.175823 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 847) took 1184 ms. Trace:
I20260709 12:25:02.139458  1851 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.176438 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 556) took 1643 ms. Trace:
I20260709 12:25:02.176702 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:02.177065 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 791) took 1231 ms. Trace:
I20260709 12:25:02.159003  1829 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:02.178145 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:02.182863 30985 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 4.
I20260709 12:25:02.183156  1829 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.183784  2187 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.184108 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1829) took 1197 ms. Trace:
I20260709 12:25:02.184477 31142 rpcz_store.cc:276] 0709 12:25:00.986875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.389743 (+402868us) service_pool.cc:224] Handling call
0709 12:25:01.390063 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.183745 (+793682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.183775 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.183817 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:02.183875 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.183942 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.184037 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":792159}
W20260709 12:25:02.178337 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 1.19484e+06us
I20260709 12:25:02.186549  2174 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.186997  2157 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:02.187572  2157 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.188731  2174 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.188920  1509 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.189442  1509 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.189771  2157 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.189846  1951 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.190701 31693 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:02.190471  1951 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:01.862015  1886 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.191486 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 559) took 1657 ms. Trace:
W20260709 12:25:02.179003 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.177547 31060 rpcz_store.cc:276] 0709 12:25:00.945187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.945606 (+   419us) service_pool.cc:224] Handling call
0709 12:25:00.960795 (+ 15189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.261654 (+300859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.261703 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:01.481295 (+219592us) log.cc:844] Serialized 26 byte log entry
0709 12:25:01.721040 (+239745us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:01.721078 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.721136 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.097630 (+376494us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:02.176788 (+ 79158us) raft_consensus.cc:1685] finished
0709 12:25:02.176809 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.177008 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162358,"raft.queue_time_us":204956,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":18486,"spinlock_wait_cycles":14976256,"thread_start_us":341757,"threads_started":2,"wal-append.queue_time_us":284493}
I20260709 12:25:02.174082  2117 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.178472 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:02.205513 30990 raft_consensus.cc:2410] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:02.152786  2054 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.170608  1474 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:02.176970 31143 rpcz_store.cc:276] 0709 12:25:00.533096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.573060 (+ 39964us) service_pool.cc:224] Handling call
0709 12:25:00.622298 (+ 49238us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.689034 (+ 66736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.689072 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.689696 (+   624us) log.cc:844] Serialized 52 byte log entry
0709 12:25:00.752715 (+ 63019us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:00.939671 (+186956us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.176125 (+1236454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.176199 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.176239 (+    40us) raft_consensus.cc:1685] finished
0709 12:25:02.176251 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.176380 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1239626,"prepare.queue_time_us":57661,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":185,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":9458,"spinlock_wait_cycles":801664,"thread_start_us":209624,"threads_started":2,"wal-append.queue_time_us":290025}
I20260709 12:25:02.196101 31065 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:02.137802  2050 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:25:02.202857 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.201380 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:02.200496 31140 rpcz_store.cc:276] 0709 12:25:00.533966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.791990 (+258024us) service_pool.cc:224] Handling call
0709 12:25:00.806949 (+ 14959us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.807307 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.807334 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.807845 (+   511us) log.cc:844] Serialized 52 byte log entry
0709 12:25:01.269026 (+461181us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:01.455231 (+186205us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.859844 (+404613us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:02.185949 (+326105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.186026 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.186064 (+    38us) raft_consensus.cc:1685] finished
0709 12:25:02.186151 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.191416 (+  5265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":746998,"prepare.queue_time_us":207421,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":732743,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":21401,"spinlock_wait_cycles":3901312,"thread_start_us":460791,"threads_started":1,"wal-append.queue_time_us":742185}
I20260709 12:25:02.176168 31690 rpcz_store.cc:276] 0709 12:25:01.044400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.064045 (+ 19645us) service_pool.cc:224] Handling call
0709 12:25:01.078067 (+ 14022us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:01.125504 (+ 47437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.125533 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:01.188393 (+ 62860us) log.cc:844] Serialized 52 byte log entry
0709 12:25:01.496924 (+308531us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:01.764718 (+267794us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.816739 (+ 52021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.816841 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.175626 (+358785us) raft_consensus.cc:1685] finished
0709 12:25:02.175644 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.175772 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34374,"prepare.queue_time_us":330230,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":111472,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":23305,"thread_start_us":570444,"threads_started":3,"wal-append.queue_time_us":444920}
W20260709 12:25:02.212487 31109 connection.cc:256] RPC call timeout handler was delayed by -1.2104952820000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 12:25:02.217419 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.179800  1968 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.212455 31142 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:02.213918 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2286) took 1250 ms. Trace:
I20260709 12:25:01.854725  2141 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.180676  2027 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.176182  2029 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:02.218590 31109 connection.cc:256] RPC call timeout handler was delayed by -1.0502952819997518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
I20260709 12:25:02.181190 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2077) took 1199 ms. Trace:
I20260709 12:25:02.218657 31139 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:02.222671 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:02.201886  1877 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.204777  2025 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.225472  2178 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:02.206873  2157 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.219552 31059 rpcz_store.cc:276] 0709 12:25:00.963865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.965640 (+  1775us) service_pool.cc:224] Handling call
0709 12:25:00.965956 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:01.060468 (+ 94512us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:01.060498 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:01.070975 (+ 10477us) log.cc:844] Serialized 52 byte log entry
0709 12:25:01.360087 (+289112us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:01.360146 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.360243 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.213667 (+853424us) raft_consensus.cc:1685] finished
0709 12:25:02.213695 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.213850 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":295787,"raft.queue_time_us":97174,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":139694,"thread_start_us":226477,"threads_started":1,"wal-append.queue_time_us":422746}
I20260709 12:25:02.236162 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:02.206923  1509 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.249514 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.231724 31141 rpcz_store.cc:276] 0709 12:25:00.981390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.096156 (+114766us) service_pool.cc:224] Handling call
0709 12:25:01.096428 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.096633 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:01.096870 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.180879 (+1084009us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.180928 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:02.180979 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.181038 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.181138 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1083780,"prepare.queue_time_us":3633,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":43,"raft.queue_time_us":1085481,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":67}
I20260709 12:25:02.238128 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:02.252851 31068 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:02.215526  2174 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.269299  2080 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.228821  2054 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.272725  2054 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.273171  2025 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.273541  2054 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:02.251494  2146 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.275231  2025 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.275476  2146 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:02.235366  2000 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.276605  2000 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.278532  2000 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.278563 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.279314 31142 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
W20260709 12:25:02.279707 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 43625.5us
I20260709 12:25:02.282559  2157 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.285199  2157 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.233618  1886 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:02.290669  1531 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:02.291404 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2025) took 1483 ms. Trace:
I20260709 12:25:02.291786 30986 rpcz_store.cc:276] 0709 12:25:00.808372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.819515 (+ 11143us) service_pool.cc:224] Handling call
0709 12:25:00.835264 (+ 15749us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.563273 (+728009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.291154 (+727881us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b540029a9d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:02.291343 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1662617984}
I20260709 12:25:02.294489  2150 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:02.217276  1950 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.258910 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.298831 31139 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:02.298740  1950 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.301009  1950 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:02.259025  2175 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.247597  1474 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:02.302781  1474 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.247519  2012 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.207787  2017 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.262455 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.308992  2012 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.309428 31143 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:02.215926  2170 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.310425  2017 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:02.310339  2170 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.269412  2174 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.311137  2012 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.312374  2170 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.305842 31661 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:02.249123  2027 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.314546  2027 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.315140  2193 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:02.315670  2193 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:02.316543  2193 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.315140 31135 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:02.265291  2068 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:02.265841  1877 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.162740 31136 rpcz_store.cc:276] 0709 12:25:00.533705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.741133 (+207428us) service_pool.cc:224] Handling call
0709 12:25:00.741498 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.741794 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:00.742183 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 12:25:00.850503 (+108320us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.607497 (+756994us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:02.153804 (+546307us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b18001ebec0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:25:02.153954 (+   150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.154005 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:02.154053 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.154112 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.154207 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1317923,"prepare.queue_time_us":17957,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":81,"raft.queue_time_us":1305071,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":12396,"spinlock_wait_cycles":53053952,"thread_start_us":107551,"threads_started":1,"wal-append.queue_time_us":193073}
I20260709 12:25:02.321174  2175 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:02.321369 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2022) took 1622 ms (client timeout 1873 ms). Trace:
W20260709 12:25:02.321772 30984 rpcz_store.cc:269] 0709 12:25:00.698861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.707003 (+  8142us) service_pool.cc:224] Handling call
0709 12:25:02.321190 (+1614187us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540029b8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:02.321315 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3244471680}
I20260709 12:25:02.323036 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:02.323645 31060 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:02.267202 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:02.163053  2069 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.281852 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:02.321663  2175 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.307683 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:02.312466  2174 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.264128  2071 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.263701  2038 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.294536  2025 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 12:25:02.332455 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50687.5us
I20260709 12:25:02.274137  2054 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.340055 31064 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:02.341282 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:02.286087  1951 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.322407  1877 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:02.353619  2170 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.367272 31068 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:02.356931 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:02.411376 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54524.6us
I20260709 12:25:02.412076 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:02.412765 31060 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:02.414721 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:02.417052 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:02.415767 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:02.418633 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.373660 31063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.029s	sys 0.060s
W20260709 12:25:02.365247 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 210116us
W20260709 12:25:02.419561 31063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.029s	sys 0.061s
I20260709 12:25:02.356509  1758 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:25:02.399927  2153 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:02.400569 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.400460  2000 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.400753 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:02.401011  1509 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:02.403187  1531 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:02.353056  2027 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:02.355899  2138 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:02.362805  2012 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:02.420956 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.413302 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:02.420742 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:02.421757 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:02.457549 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.427374 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:02.399367 31064 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
W20260709 12:25:02.474730 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 53037.7us
I20260709 12:25:02.475140 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:02.475934  2017 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:02.422356  2038 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:02.421492  2025 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:02.483013 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:02.477837 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 262026us
W20260709 12:25:02.483047 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.484333 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:02.466518  2014 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:02.425976  1951 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:02.426209  2130 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:02.488194 31058 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:02.416934  2170 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.486049  2025 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.504984 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:02.404454  2197 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:25:02.412125  2187 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.468220  2153 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:02.492599 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 339539us
I20260709 12:25:02.477658  2050 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:02.466753  1877 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
W20260709 12:25:02.504848  1358 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.000s
I20260709 12:25:02.550518  1358 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.552206  1358 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:02.482457  2148 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:02.498857  1474 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:02.518311 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.514999 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:02.553337  1474 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:02.526278 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:02.553773  1474 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:02.537112 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.527566 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.548308 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:02.489885 30888 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 5 to 6, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.488176  1509 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.549409  2050 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.552850  2148 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:02.552887 31691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 842) took 2135 ms (client timeout 2077 ms). Trace:
I20260709 12:25:02.489161  2014 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.558234 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:02.563108 30888 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 6 to 7, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 7 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.462648 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:02.478826  2000 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.574054  2211 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:02.500533  2038 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:02.480794  2012 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:02.502379  1951 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:02.516146  2025 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:02.884217 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.582544 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 564) took 2005 ms. Trace:
W20260709 12:25:02.761022 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 284033us
I20260709 12:25:03.000347 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.980224 31136 raft_consensus.cc:606] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:02.879153 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:02.645025 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 231726us
W20260709 12:25:02.668151 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.575294 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:02.856195 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:02.498858  2208 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:03.005492 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:02.973694 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:03.000458 31691 rpcz_store.cc:269] 0709 12:25:00.417117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.417245 (+   128us) service_pool.cc:224] Handling call
0709 12:25:02.552686 (+2135441us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400514d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:02.552819 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3191191936}
I20260709 12:25:03.003162 31136 raft_consensus.cc:614] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:02.811335 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.011718 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.970587 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:03.013537 31139 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:25:02.515316  2130 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:02.856694  1886 consensus_meta.cc:287] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.000s
I20260709 12:25:02.782881 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.029256 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:03.029956 31690 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:25:03.030076 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.013957 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.015121 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:02.970721 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:03.031003 31143 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:03.017944 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:02.819710  2214 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:02.982076 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:02.778929 30889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 318) took 24301 ms (client timeout 15000 ms). Trace:
W20260709 12:25:03.020524 31061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.005s	sys 0.009s
W20260709 12:25:02.958967  2193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.002s	sys 0.012s
W20260709 12:25:03.039641 31061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.005s	sys 0.009s
I20260709 12:25:02.960780  2027 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:25:02.582680  2186 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:25:02.785609  2199 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 12:25:03.015702 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:02.986951  2014 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:03.037250 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:03.080694 30992 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:02.824519  2138 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:03.028652 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:03.019886 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:03.040411 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.759244  2017 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.042488 31696 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:03.043498 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:02.821563  1509 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:02.947877  2069 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:03.026726 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 468562us
I20260709 12:25:02.576949  2080 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:03.001103 31138 rpcz_store.cc:276] 0709 12:25:00.576933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.853918 (+276985us) service_pool.cc:224] Handling call
0709 12:25:00.854244 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.156111 (+1301867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.156140 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.162565 (+  6425us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.478151 (+315586us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:02.478204 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.478311 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.582295 (+103984us) raft_consensus.cc:1685] finished
0709 12:25:02.582318 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.582472 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1322391,"raft.queue_time_us":1304466,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":10588,"thread_start_us":296403,"threads_started":1,"wal-append.queue_time_us":396950}
I20260709 12:25:02.574443  2054 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:02.691114  2179 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:02.823077  2170 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:03.042501 31697 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:03.076221 30983 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:25:03.082162 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.086633 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 58394.5us
W20260709 12:25:03.042586  2193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.003s	sys 0.012s
I20260709 12:25:03.139855 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.179253 31691 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:03.016537  2208 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:03.012712  2217 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:03.146670 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.164451 30888 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 5, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 5 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:03.166306 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:03.042127  1758 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.017777  1965 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.167238 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:03.138718 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 137322us
I20260709 12:25:03.169178 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:03.171105 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.171209 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:03.059156  2196 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:03.171685 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.084774  2207 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:02.821915  2153 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:03.182423 31694 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.095218  2175 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:03.029377  1474 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:03.184187 31106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.017s	sys 0.017s
W20260709 12:25:03.232547 31106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.017s	sys 0.020s
W20260709 12:25:03.234300 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 806941us
I20260709 12:25:03.095714  2027 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:03.254868 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 83946.3us
W20260709 12:25:03.252462 31033 leader_election.cc:336] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.873s (SENT)
I20260709 12:25:03.281647 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:03.207916  2146 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.913s	user 0.013s	sys 0.000s
I20260709 12:25:03.180183  2138 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.290911 31692 raft_consensus.cc:2410] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:03.177707  2187 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:03.185880  2205 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.187182  2208 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:03.300761 30888 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 5, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 5 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:03.281373 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:03.167289  2017 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.278685 30984 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:03.185366  2080 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.287611 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.287853 31140 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:25:03.300019  2175 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:03.300211 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:03.300633 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48192.6us
W20260709 12:25:03.307866 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52561.8us
I20260709 12:25:03.280738  2211 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:03.311926 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:03.316458 31696 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:25:03.177004  2179 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.320374 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.322268 31690 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:03.327594 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.311040 30888 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 3 to 4, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:03.206959  1758 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.195212  1965 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.333349 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.334586 31064 raft_consensus.cc:2410] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:25:03.335364 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 157117us
I20260709 12:25:03.336562 31698 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:03.338307 31061 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:03.338735 31061 pending_rounds.cc:85] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:25:03.338969 31061 pending_rounds.cc:107] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:03.339246 31061 raft_consensus.cc:2889] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:03.340969 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.341857 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.342672 31064 raft_consensus.cc:2410] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:25:03.340322 31033 leader_election.cc:336] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.200s (SENT)
I20260709 12:25:03.343851 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.344718 31063 raft_consensus.cc:2410] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:03.345163 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:03.343742 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:03.346254 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.347009 31058 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:25:03.350252 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.354456 31695 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.867s	user 0.010s	sys 0.000s
I20260709 12:25:03.354923 31695 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:03.356115 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.187554  2130 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.357102 31690 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:03.183136  2148 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.363880 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.251343  1951 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.329706  1950 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.281543  2217 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.302656  2138 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:03.251145  2038 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.285140  2207 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:03.321635  2080 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.312345  2205 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:03.555168 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.558919  2141 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.561430  2190 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.575675  2190 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.363957 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.379004 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:03.624958 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.556497 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 535) took 1361 ms. Trace:
W20260709 12:25:03.040730 30889 rpcz_store.cc:269] 0709 12:24:38.241664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.247136 (+  5472us) service_pool.cc:224] Handling call
0709 12:25:02.515105 (+24267969us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:52.433907 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:52.434695 (+   788us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:52.434755 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:24:52.434775 (+    20us) tablet.cc:662] Decoding operations
0709 12:24:52.623078 (+188303us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:52.631993 (+  8915us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:52.637029 (+  5036us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:24:53.209930 (+572901us) tablet.cc:701] Row locks acquired
0709 12:24:53.216991 (+  7061us) write_op.cc:260] PREPARE: finished
0709 12:24:53.217175 (+   184us) write_op.cc:270] Start()
0709 12:24:53.217337 (+   162us) write_op.cc:276] Timestamp: P: 1783599893217111 usec, L: 0
0709 12:24:53.217360 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:24:53.599147 (+381787us) log.cc:844] Serialized 6280 byte log entry
0709 12:24:54.128906 (+529759us) op_driver.cc:464] REPLICATION: finished
0709 12:24:54.391708 (+262802us) write_op.cc:301] APPLY: starting
0709 12:24:54.391820 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:54.392189 (+   369us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:54.392359 (+   170us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:54.402452 (+ 10093us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:54.402814 (+   362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 12:24:54.402861 (+    47us) write_op.cc:312] APPLY: finished
0709 12:24:54.403282 (+   421us) log.cc:844] Serialized 113 byte log entry
0709 12:24:54.403497 (+   215us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:54.403663 (+   166us) write_op.cc:454] Released schema lock
0709 12:24:54.403731 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:55.019732 (+616001us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88fe28 00007f2eef2ca581 00007f2eef2c9d58 00007f2eef2ce9f1 00007f2eef2ce9a9 00007f2eef2ce939 00007f2eef2ce901 00007f2eef2cdb2d 0000000000388634 0000000000388479
0709 12:24:55.019933 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:55.019961 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20841,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":262501,"apply.run_cpu_time_us":4408,"apply.run_wall_time_us":12429,"num_ops":16,"prepare.queue_time_us":453406,"prepare.run_cpu_time_us":50852,"prepare.run_wall_time_us":2586025,"raft.queue_time_us":240913,"raft.run_cpu_time_us":10512,"raft.run_wall_time_us":171507,"replication_time_us":911376,"spinlock_wait_cycles":309302528,"thread_start_us":553890,"threads_started":4,"wal-append.queue_time_us":116368,"wal-append.run_cpu_time_us":23657,"wal-append.run_wall_time_us":1658829}]]}
I20260709 12:25:03.565312 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1998) took 1226 ms. Trace:
I20260709 12:25:03.332994  2187 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.629393 31692 rpcz_store.cc:276] 0709 12:25:02.195504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.195644 (+   140us) service_pool.cc:224] Handling call
0709 12:25:03.556276 (+1360632us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540034fdd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:03.556436 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2466981632}
I20260709 12:25:03.451906 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:03.626175 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 295204us
W20260709 12:25:03.629022 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.364460 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:03.515909 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 204040us
I20260709 12:25:03.630077 31060 rpcz_store.cc:276] 0709 12:25:02.338900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.460481 (+121581us) service_pool.cc:224] Handling call
0709 12:25:02.460928 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.461373 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.461415 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.468117 (+  6702us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.902159 (+434042us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:02.902210 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.902307 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.565008 (+662701us) raft_consensus.cc:1685] finished
0709 12:25:03.565028 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.565252 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4669,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8397,"thread_start_us":399054,"threads_started":1,"wal-append.queue_time_us":972813}
I20260709 12:25:03.639194 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.455214 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.582688 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:03.362510 30888 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:03.463657 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:03.581561 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:03.628949 31059 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:03.645978 31141 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.646353 31063 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.646929 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.647725 31059 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:03.640156 31060 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:03.636634 31061 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:03.649968 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:03.640300 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:03.651448 31060 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:03.554884 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.637904 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:03.650846 31139 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:25:03.479605  2222 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 12:25:03.651639 31144 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:03.653834 31060 pending_rounds.cc:85] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 3
I20260709 12:25:03.654760 31060 pending_rounds.cc:107] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:25:03.649262 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:03.649379 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:03.473920  2200 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.461s	user 0.000s	sys 0.001s
I20260709 12:25:03.578626  2027 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:03.656056 31059 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:03.655346 31060 raft_consensus.cc:2889] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:03.480120  2223 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 12:25:03.655635 31143 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:25:03.643178 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:03.659073 31058 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.657881 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:03.426323  2203 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.445s	user 0.001s	sys 0.000s
W20260709 12:25:03.480822  2221 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 12:25:03.560087  2227 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.218s	user 0.000s	sys 0.000s
I20260709 12:25:03.358043  2175 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:03.669270 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.563408  2174 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:03.480631  2232 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:03.559583  1758 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:03.666235 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:03.666299 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:03.467816  2206 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.425s	user 0.000s	sys 0.001s
I20260709 12:25:03.661612 31060 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:25:03.552374  1531 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.561978  1965 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:03.481796  2000 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:03.644171  2213 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.601s	user 0.006s	sys 0.002s
I20260709 12:25:03.634733  2187 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:03.480305  2202 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:25:03.656431  2241 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:25:03.677023 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:03.680202 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:03.712545 31065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.041s	sys 0.079s
W20260709 12:25:03.712960 31065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.041s	sys 0.079s
I20260709 12:25:03.713222 31139 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.719213 31058 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:03.687124 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.687207 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.682188 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1990) took 1854 ms. Trace:
I20260709 12:25:03.681519 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.693917  2174 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:03.710141  2187 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.672696  2170 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:03.657840  1950 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:03.717571 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:03.680281 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:03.691175 30985 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.003s	sys 0.015s
I20260709 12:25:03.691375 30888 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 2 to 5, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 5 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:03.694675  1758 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:03.724191 31062 rpcz_store.cc:276] 0709 12:25:01.827434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.827620 (+   186us) service_pool.cc:224] Handling call
0709 12:25:01.848840 (+ 21220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.109411 (+260571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.109448 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.126426 (+ 16978us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.477526 (+351100us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:02.477564 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.477626 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.681933 (+1204307us) raft_consensus.cc:1685] finished
0709 12:25:03.681960 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.682115 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25985,"raft.queue_time_us":231173,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":17171,"thread_start_us":319998,"threads_started":1,"wal-append.queue_time_us":713634}
I20260709 12:25:03.694540  2232 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:03.674615  2175 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:03.734041 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 69858.7us
W20260709 12:25:03.868193 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.738996 31692 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:03.721647  2179 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:03.744367 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.726346  2207 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:03.711725  2017 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:03.747907  2025 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.000s
W20260709 12:25:03.749007 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 108763us
I20260709 12:25:03.749768 31063 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:03.752840  2131 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.754439 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 912) took 1011 ms. Trace:
W20260709 12:25:03.760466 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.762162  2071 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:03.773943  1851 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:03.780514 31664 connection.cc:256] RPC call timeout handler was delayed by -1.6843010689999574s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:25:03.894253 31664 connection.cc:256] RPC call timeout handler was delayed by -1.385501068999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
W20260709 12:25:03.894723 31664 connection.cc:256] RPC call timeout handler was delayed by -1.1182010689999515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 12:25:03.886377 31060 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
W20260709 12:25:03.783183  2225 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.318s	user 0.002s	sys 0.000s
I20260709 12:25:03.793056 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 911) took 1064 ms. Trace:
I20260709 12:25:03.898586 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:03.794431  2220 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.464s	user 0.002s	sys 0.000s
I20260709 12:25:03.797045  2050 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:03.804177  1474 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:03.814477 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.911312 31058 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:25:03.817032  2210 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.057s	user 0.002s	sys 0.000s
I20260709 12:25:03.822906  2012 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:03.823907 31135 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:03.824627 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60b6187f32774daba89763099dabeb2a"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:03.927404 31695 raft_consensus.cc:606] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:03.826982 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 563) took 3155 ms. Trace:
I20260709 12:25:03.790126 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:03.692819  1531 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.718276  2000 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:03.854007 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:03.854203 31061 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:03.679383  2236 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 12:25:03.857038 30985 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.003s	sys 0.015s
W20260709 12:25:03.857652 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 140138us
I20260709 12:25:03.865773 31139 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:03.728495  2174 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:03.878327 30888 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 6 to 7, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 7 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:03.897879 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.899451 31141 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:03.898994 31698 rpcz_store.cc:276] 0709 12:25:02.728526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.728684 (+   158us) service_pool.cc:224] Handling call
0709 12:25:02.759899 (+ 31215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:03.336931 (+577032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.792771 (+455840us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b540055c654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:03.792987 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325370112}
I20260709 12:25:03.956138 31106 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:03.895135 31664 connection.cc:256] RPC call timeout handler was delayed by -1.1002010689999224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2163s for a response
I20260709 12:25:03.900244 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.957185 31060 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:03.903038 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 1997) took 1570 ms. Trace:
W20260709 12:25:03.962343 31664 connection.cc:256] RPC call timeout handler was delayed by -1.0714010689998759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2195s for a response
I20260709 12:25:03.907467 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.875387  1965 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:25:03.910506 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:03.970605 31059 raft_consensus.cc:2410] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:03.962437 31068 rpcz_store.cc:276] 0709 12:25:02.332159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.418298 (+ 86139us) service_pool.cc:224] Handling call
0709 12:25:02.418665 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.419100 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:02.903685 (+484585us) log.cc:844] Serialized 14 byte log entry
0709 12:25:03.005616 (+101931us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:03.013425 (+  7809us) log.cc:844] Serialized 26 byte log entry
0709 12:25:03.013581 (+   156us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:03.013611 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.013675 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.902866 (+889191us) raft_consensus.cc:1685] finished
0709 12:25:03.902883 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.902989 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11455,"prepare.queue_time_us":84628,"prepare.run_cpu_time_us":32423,"prepare.run_wall_time_us":347251,"raft.queue_time_us":1785,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":32840,"thread_start_us":535816,"threads_started":2,"wal-append.queue_time_us":261998}
I20260709 12:25:03.914937 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:03.934161 31690 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:03.973079 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:03.875828  1995 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.900871 31697 rpcz_store.cc:276] 0709 12:25:02.743106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.782404 (+ 39298us) service_pool.cc:224] Handling call
0709 12:25:03.754239 (+971835us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b5400738cd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:03.754373 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29240576}
I20260709 12:25:03.906826  2190 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:03.972855 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:03.972926 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:03.973711 30989 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.981232 30990 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.928035 31137 rpcz_store.cc:276] 0709 12:25:00.576683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.851948 (+275265us) service_pool.cc:224] Handling call
0709 12:25:00.872623 (+ 20675us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:02.173067 (+1300444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.173095 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:02.173740 (+   645us) log.cc:844] Serialized 78 byte log entry
0709 12:25:02.498231 (+324491us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:02.498290 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.498454 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.731543 (+1233089us) raft_consensus.cc:1685] finished
0709 12:25:03.731562 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.731694 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1262479,"raft.queue_time_us":1268705,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":183,"thread_start_us":324057,"threads_started":1,"wal-append.queue_time_us":603982}
I20260709 12:25:03.982028 30983 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
W20260709 12:25:03.954073  2233 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.451s	user 0.001s	sys 0.000s
I20260709 12:25:03.970249 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:03.925865  1951 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:03.968528 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 330705us
I20260709 12:25:03.977813 31059 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:25:03.978451 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.976665 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:03.986784 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:03.989753 31694 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.736s	user 0.006s	sys 0.000s
I20260709 12:25:03.897272  2131 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.979059 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:03.893172  2025 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:03.984490 30983 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:03.900139  2170 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:03.909025  1851 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:25:03.903782  2071 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:03.990200 31694 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:04.006347 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 914) took 1064 ms. Trace:
I20260709 12:25:03.992498 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:03.993671 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 78822.2us
I20260709 12:25:03.963606  2174 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.955183  2054 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:03.980238  1474 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:03.936635  2205 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:03.998378 30986 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:25:03.998378 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:04.003459 30989 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:03.980646  1995 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.011193 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:04.007782 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:04.016173 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 81140.2us
I20260709 12:25:03.969444  2138 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:03.995623 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:03.974745  2218 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:04.020426 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:04.006680 31694 rpcz_store.cc:276] 0709 12:25:02.942200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.077227 (+135027us) service_pool.cc:224] Handling call
0709 12:25:04.006293 (+929066us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:03.997924  2017 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.997845  2179 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:03.963717  2229 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.340s
I20260709 12:25:03.981264  2217 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:03.998756  1951 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.031050 30990 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:04.058655 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2288) took 1869 ms. Trace:
W20260709 12:25:04.065407 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57683.7us
I20260709 12:25:04.072901 30986 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:04.033766 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:04.102936 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:04.103829 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:04.000232  2241 raft_consensus.cc:2764] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:25:04.103003 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 69323.4us
I20260709 12:25:04.009363  2080 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:04.050765 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:04.029457 30888 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 5, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 5 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
W20260709 12:25:04.018127  2239 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 12:25:04.110740 31135 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76399531471a4790a56d52a6ea052864"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:04.101590 30989 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:04.098017 31067 rpcz_store.cc:276] 0709 12:25:02.189124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.941443 (+752319us) service_pool.cc:224] Handling call
0709 12:25:02.979028 (+ 37585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.979392 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.979420 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.979862 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:25:03.174552 (+194690us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:03.174595 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.174677 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.782562 (+607885us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:04.030665 (+248103us) raft_consensus.cc:1685] finished
0709 12:25:04.030689 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.058574 (+ 27885us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3014,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":16201,"spinlock_wait_cycles":69538560,"thread_start_us":194263,"threads_started":1,"wal-append.queue_time_us":359066}
I20260709 12:25:04.112967 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:04.076011  2220 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:25:04.046434  2208 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.000s
I20260709 12:25:04.118340 31135 raft_consensus.cc:606] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:04.254189 31135 raft_consensus.cc:614] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:04.254518  1358 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:04.255170  1358 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:04.023847  1474 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.260139 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2090) took 1311 ms. Trace:
W20260709 12:25:04.256734 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 206073us
I20260709 12:25:04.261746 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:04.145563 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.099303  2179 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:04.252658  2255 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:25:04.272017  1358 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.260512 31142 rpcz_store.cc:276] 0709 12:25:02.948541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.948817 (+   276us) service_pool.cc:224] Handling call
0709 12:25:02.987243 (+ 38426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:03.063847 (+ 76604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.063893 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:03.166329 (+102436us) log.cc:844] Serialized 26 byte log entry
0709 12:25:03.333902 (+167573us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:03.333987 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.334136 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.138837 (+804701us) raft_consensus.cc:1685] finished
0709 12:25:04.138868 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.260083 (+121215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19071,"raft.queue_time_us":182705,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":3717,"thread_start_us":177581,"threads_started":2,"wal-append.queue_time_us":303928}
I20260709 12:25:04.026059  2076 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:04.266115 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2273) took 1247 ms. Trace:
W20260709 12:25:04.186278 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.021793  2000 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:04.252817 30888 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 5 to 6, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:04.075281  2029 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:04.137882  1889 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:04.274816 30988 rpcz_store.cc:276] 0709 12:25:03.018734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.018847 (+   113us) service_pool.cc:224] Handling call
0709 12:25:03.075108 (+ 56261us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:03.084297 (+  9189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.084324 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:03.098751 (+ 14427us) log.cc:844] Serialized 52 byte log entry
0709 12:25:03.625884 (+527133us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:03.625922 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.626000 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.265917 (+639917us) raft_consensus.cc:1685] finished
0709 12:25:04.265934 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.266055 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2581,"raft.queue_time_us":71103,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10557,"thread_start_us":418361,"threads_started":1,"wal-append.queue_time_us":486015}
W20260709 12:25:04.252751  2251 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.262s	user 0.000s	sys 0.000s
W20260709 12:25:04.273097  2234 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.414s	user 0.001s	sys 0.001s
I20260709 12:25:04.252247  2012 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.027169  1995 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.116262  2236 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:04.266188  1474 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:04.306331 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.307029  1951 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:04.307574 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.307821 30986 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:04.308270 30988 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:04.276312  2211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.006s	sys 0.023s
W20260709 12:25:04.308915  2211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.007s	sys 0.024s
I20260709 12:25:04.309185  2211 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:04.023262  2232 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.296563 30888 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 5, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 5 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:04.276979  2179 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.302407 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:04.283558  2238 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 12:25:04.305838  2236 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:04.277448  2000 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.305886 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:04.277304  2076 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.137643  2047 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.156s
I20260709 12:25:04.108841  1950 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:04.310084  2211 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.118165  2138 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:04.101747  2218 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.336845 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2272) took 1320 ms. Trace:
I20260709 12:25:04.338635 31141 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:04.366114 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.367201 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:04.367904 31141 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.341663 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:04.370110 31144 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:04.343984 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 67875.8us
I20260709 12:25:04.062448  2205 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.239s
I20260709 12:25:04.350720 31142 raft_consensus.cc:606] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:04.350378 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 326028us
I20260709 12:25:04.350859 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:04.382283 31135 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.382678 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:04.350415 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:04.354951 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:04.380335 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 382019us
I20260709 12:25:04.276787  2208 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:04.382175 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:04.393221 31143 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:04.386525 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 273587us
I20260709 12:25:04.384583 31698 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 5.
I20260709 12:25:04.387239 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:04.387776 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.388325 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.395368 31060 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:04.385771 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:04.365482 30987 rpcz_store.cc:276] 0709 12:25:03.016191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.016327 (+   136us) service_pool.cc:224] Handling call
0709 12:25:03.016612 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:03.024883 (+  8271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.024911 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:03.030093 (+  5182us) log.cc:844] Serialized 26 byte log entry
0709 12:25:03.667990 (+637897us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:03.668301 (+   311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.668380 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.336646 (+668266us) raft_consensus.cc:1685] finished
0709 12:25:04.336665 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.336789 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70868,"raft.queue_time_us":8025,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":49585,"thread_start_us":498540,"threads_started":1,"wal-append.queue_time_us":762477}
I20260709 12:25:04.306099  2254 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:04.116065  2244 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:04.381687 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.457939 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:04.140657  2235 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.215s
I20260709 12:25:04.463353 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.352922  1889 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:04.353207  2236 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:04.346328  2029 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.305078  2012 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.484440 31138 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:04.384060  2138 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.362895  2076 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.364203  2000 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:04.468652 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 73349.5us
I20260709 12:25:04.489524 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:04.349290  2249 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:04.473668 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:04.430259  2025 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:04.478529 30987 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:04.490869 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.349493  2226 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.445s
I20260709 12:25:04.528275 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:04.466732  2190 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.534646 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.535872 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 819) took 1528 ms. Trace:
I20260709 12:25:04.536368 31065 rpcz_store.cc:276] 0709 12:25:03.007125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.007285 (+   160us) service_pool.cc:224] Handling call
0709 12:25:03.036978 (+ 29693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:03.037167 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.037190 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:03.091478 (+ 54288us) log.cc:844] Serialized 26 byte log entry
0709 12:25:03.713382 (+621904us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:03.713438 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.713531 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.535605 (+822074us) raft_consensus.cc:1685] finished
0709 12:25:04.535632 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.535796 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":536009,"threads_started":1,"wal-append.queue_time_us":559747}
I20260709 12:25:04.496829 31139 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.499917 31135 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:04.500519 31141 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:04.539203 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:04.539924 31141 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
W20260709 12:25:04.462831  2148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.007s	sys 0.024s
I20260709 12:25:04.528956 31142 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:04.469432  2131 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:04.459062  2254 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.552245 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:04.552968 31135 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:04.459441  2170 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:04.533933 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.464350  2244 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:04.473747  2071 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:04.544672 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:04.545436 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:04.547192 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.577306 31692 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:04.547467 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:04.473681  1509 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:04.491592  2138 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:04.495638 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.554360 30987 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:04.554978 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:04.504786  2080 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:04.582962 30988 raft_consensus.cc:2368] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
W20260709 12:25:04.548252  2219 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:25:04.548360  2148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.007s	sys 0.024s
I20260709 12:25:04.584662 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:04.553104  2256 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.455s	user 0.001s	sys 0.000s
I20260709 12:25:04.497936  2130 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:25:04.490495  2029 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:04.598793 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:04.646767 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.583218  1877 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.569603  2254 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.576231  2244 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.655006 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.590019  2131 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:04.599357  1889 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:04.549571  2190 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.667160 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.665512 31139 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
W20260709 12:25:04.671868 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.672187 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 182760us
I20260709 12:25:04.590544  1509 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.605140  2192 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:25:04.681437 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.752334 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.839032 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.839360  2208 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:04.840097 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.774278 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2004) took 1418 ms. Trace:
I20260709 12:25:04.841112 31064 rpcz_store.cc:276] 0709 12:25:03.355519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.652708 (+297189us) service_pool.cc:224] Handling call
0709 12:25:03.653070 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:03.882719 (+229649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.882751 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:03.883208 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:25:03.976038 (+ 92830us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:03.976104 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.099637 (+123533us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540027c254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:04.099697 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.681934 (+582237us) raft_consensus.cc:1685] finished
0709 12:25:04.681959 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.774175 (+ 92216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103446,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6446,"spinlock_wait_cycles":500545408,"thread_start_us":92416,"threads_started":1,"wal-append.queue_time_us":143808}
W20260709 12:25:04.816249 31662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.004s	sys 0.003s
W20260709 12:25:04.843986 31662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.004s	sys 0.003s
W20260709 12:25:04.739027 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.752266 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.819325 31065 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:04.856287 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.860419 30988 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:04.862862 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:04.863392 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:04.864356 31060 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:04.865406 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:04.866008 31065 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:04.866938 30987 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
I20260709 12:25:04.815313 31135 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:04.772112 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.004s	sys 0.003s
W20260709 12:25:04.867758 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.004s	sys 0.003s
W20260709 12:25:04.868060 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 764244us
I20260709 12:25:04.869082 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:04.869861  2241 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:04.870271  2241 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.870646  2241 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:04.873986  2014 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.707s	user 0.012s	sys 0.005s
I20260709 12:25:04.874446  2014 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:04.875658 31136 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:04.876812  2014 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:04.877345 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:04.878058 31065 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:04.906356 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 520602us
W20260709 12:25:04.797479  2267 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 12:25:04.798274  2261 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 12:25:04.911213 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 95480.9us
W20260709 12:25:04.922587 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 50865.4us
I20260709 12:25:04.887079 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:04.923365 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:04.924502 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:04.925781 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:04.656177  2269 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:04.922919 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:04.927721 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.948550 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:05.010838 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:04.928576 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:05.013569 31696 raft_consensus.cc:2410] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:05.014683 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:04.655798  1877 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:04.839067  2218 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:05.041940 31065 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
W20260709 12:25:05.034476  2262 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.483s	user 0.000s	sys 0.001s
W20260709 12:25:05.016206 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 527395us
W20260709 12:25:05.021340 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 92651.5us
W20260709 12:25:05.036201  2247 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.963s	user 0.001s	sys 0.000s
I20260709 12:25:05.036989 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 565) took 1456 ms. Trace:
I20260709 12:25:05.045825 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:04.920799  1889 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:05.038542  2276 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:05.038792  2249 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:05.040215  2264 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.392s	user 0.000s	sys 0.000s
W20260709 12:25:05.037904  2272 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:25:05.052598 31138 rpcz_store.cc:276] 0709 12:25:03.580217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.582282 (+  2065us) service_pool.cc:224] Handling call
0709 12:25:03.648648 (+ 66366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.484840 (+836192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.036776 (+551936us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54001ea5d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:05.036934 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921695104}
W20260709 12:25:05.047710 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 119193us
I20260709 12:25:05.057296 31691 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:25:05.058508 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 135656us
I20260709 12:25:05.058022  2241 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.060880  2279 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:05.044513  2254 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.076910 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:05.074179 31106 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:05.071260 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:05.081912 31696 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:05.082480  2279 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.083761  2279 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:05.084157  2254 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:05.084723  2254 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:05.085328  2254 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:05.041327  2029 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:05.069873  2047 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:05.087911  2047 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:05.082721  2071 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:05.083209  2250 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.639s	user 0.001s	sys 0.000s
I20260709 12:25:05.016588  2131 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:05.011662  2269 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.088300  2047 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:05.139768  2262 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:25:05.086205  2174 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.060s	user 0.005s	sys 0.001s
I20260709 12:25:05.106889  2218 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:05.096879  2012 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:05.173604 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 107585us
I20260709 12:25:05.107338  2071 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:05.151654  2187 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.299s	user 0.010s	sys 0.000s
W20260709 12:25:05.169615  2141 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.594s	user 0.005s	sys 0.016s
W20260709 12:25:05.166903  2261 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.090s	user 0.000s	sys 0.006s
I20260709 12:25:05.146289  2269 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:05.180102  2218 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:05.205739 31144 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.824s	user 0.010s	sys 0.006s
I20260709 12:25:05.211012 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 604) took 1235 ms. Trace:
I20260709 12:25:05.211094 31144 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:05.148114  2241 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:05.151978  2287 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:05.205675 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:05.154409  2277 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:05.196134  2071 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:05.263880 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 58257.7us
I20260709 12:25:05.211431 30985 rpcz_store.cc:276] 0709 12:25:03.975801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.975897 (+    96us) service_pool.cc:224] Handling call
0709 12:25:03.991451 (+ 15554us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:04.022469 (+ 31018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.022498 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:04.031562 (+  9064us) log.cc:844] Serialized 52 byte log entry
0709 12:25:04.458134 (+426572us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:04.650165 (+192031us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.910504 (+260339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.912157 (+  1653us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.210786 (+298629us) raft_consensus.cc:1685] finished
0709 12:25:05.210805 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.210944 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":328461,"prepare.queue_time_us":691021,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":294477,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":46088,"thread_start_us":465513,"threads_started":2,"wal-append.queue_time_us":442436}
I20260709 12:25:05.213510 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:05.154953  2263 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:25:05.270078 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 56665.7us
I20260709 12:25:05.270820 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 607) took 1274 ms. Trace:
I20260709 12:25:05.270998 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:05.207732 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 259541us
I20260709 12:25:05.181138  2207 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:05.276058 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:05.266342 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:05.205271  2190 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:05.271345 30983 rpcz_store.cc:276] 0709 12:25:03.996839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.996957 (+   118us) service_pool.cc:224] Handling call
0709 12:25:03.997237 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.022962 (+ 25725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.023000 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.029315 (+  6315us) log.cc:844] Serialized 26 byte log entry
0709 12:25:04.398543 (+369228us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:04.468532 (+ 69989us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.468748 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.468821 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.270568 (+801747us) raft_consensus.cc:1685] finished
0709 12:25:05.270595 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.270747 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27448,"prepare.queue_time_us":74048,"prepare.run_wall_time_us":46,"raft.queue_time_us":35378,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":28221,"spinlock_wait_cycles":41684736,"thread_start_us":254485,"threads_started":1,"wal-append.queue_time_us":454456}
W20260709 12:25:05.281939 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.282924 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:05.209685  2074 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.277236 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:05.146763  1889 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.214453  2218 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:05.298625  1965 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:05.303730 31136 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 568) took 1710 ms (client timeout 2201 ms). Trace:
W20260709 12:25:05.304155 31136 rpcz_store.cc:269] 0709 12:25:03.581204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.581313 (+   109us) service_pool.cc:224] Handling call
0709 12:25:05.291752 (+1710439us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400062a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:05.291922 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2826696448}
I20260709 12:25:05.275031  2260 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:05.276059  2283 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:05.275271  2207 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:05.298297  2252 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:05.169976  2174 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.306033 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:05.292035  2146 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:05.288945  1509 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:05.295410  2292 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:25:05.318208  1965 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:05.320652  2289 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:05.146848  2249 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:05.304755  2076 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.778s	user 0.011s	sys 0.006s
I20260709 12:25:05.283102  2277 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:05.330241 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 191506us
I20260709 12:25:05.293030  2190 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:05.336989 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 59819us
I20260709 12:25:05.205121  2187 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:05.406144 31143 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.012s	user 0.006s	sys 0.009s
I20260709 12:25:05.430168 31143 raft_consensus.cc:2410] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:25:05.405745  2287 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:05.405076  2264 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:25:05.319842  1851 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:05.418473 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 112547us
I20260709 12:25:05.276757  2278 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.132s
W20260709 12:25:05.409813  2179 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.021s	user 0.008s	sys 0.007s
I20260709 12:25:05.299780  2074 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.329830  2252 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.431890 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1846) took 1073 ms. Trace:
I20260709 12:25:05.318524  2260 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:05.433009 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:05.437078 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1834) took 1097 ms. Trace:
I20260709 12:25:05.460783 31143 rpcz_store.cc:276] 0709 12:25:04.339621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.339752 (+   131us) service_pool.cc:224] Handling call
0709 12:25:05.436994 (+1097242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.439754 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:05.444180 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:05.445860 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:05.446429 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:05.463083 31062 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
W20260709 12:25:05.452414 31689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 536) took 2961 ms (client timeout 1751 ms). Trace:
I20260709 12:25:05.428864  2254 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:05.429386  2190 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:05.456979 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 176346us
I20260709 12:25:05.458456 31138 rpcz_store.cc:276] 0709 12:25:04.358336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.068963 (+710627us) service_pool.cc:224] Handling call
0709 12:25:05.087614 (+ 18651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.087860 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:05.088189 (+   329us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.431560 (+343371us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.431609 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:05.431656 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.431722 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.431833 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3055,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":403,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6784,"thread_start_us":342575,"threads_started":1}
I20260709 12:25:05.416005  2141 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:05.463828 31689 rpcz_store.cc:269] 0709 12:25:02.491026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.491185 (+   159us) service_pool.cc:224] Handling call
0709 12:25:05.452236 (+2961051us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b54005564d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:05.452350 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4794981632}
I20260709 12:25:05.466470 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:05.460819 31698 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:05.426512  2300 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:05.456338 30984 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:05.414469  2146 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.441088  1877 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:05.525871 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:05.509739  1358 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:05.475077  2190 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:05.461405  2252 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:05.531386  1358 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:05.461932  2244 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.796s	user 0.000s	sys 0.021s
I20260709 12:25:05.434423  2076 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.467593  2260 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:05.426800  2297 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:25:05.547672 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:05.548239 31691 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:05.476058  2294 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.464627  2241 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:05.470429  1851 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:05.542471 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:05.613180 31062 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:05.478935  2265 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.459486  2179 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.525050  2287 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:05.617935 31033 connection.cc:256] RPC call timeout handler was delayed by -1.7496955819997311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 12:25:05.622424 31033 connection.cc:256] RPC call timeout handler was delayed by -1.7280955819996962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
I20260709 12:25:05.622867 31064 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [6.4-8.5]
W20260709 12:25:05.622870 31033 connection.cc:256] RPC call timeout handler was delayed by -1.6443955820000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 12:25:05.623571 31033 connection.cc:256] RPC call timeout handler was delayed by -1.556195581999873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
W20260709 12:25:05.623947 31033 connection.cc:256] RPC call timeout handler was delayed by -1.5426955819998511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 12:25:05.624449 31033 connection.cc:256] RPC call timeout handler was delayed by -1.517764817999705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 12:25:05.624838 31033 connection.cc:256] RPC call timeout handler was delayed by -1.5030955819997871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
I20260709 12:25:05.535918  2254 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:05.453402  2207 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:05.625208 31033 connection.cc:256] RPC call timeout handler was delayed by -1.4043648179995216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
I20260709 12:25:05.579842  1358 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:05.629892 31033 connection.cc:256] RPC call timeout handler was delayed by -1.3647648179999123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 12:25:05.630391 31033 connection.cc:256] RPC call timeout handler was delayed by -1.3647648179999123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 12:25:05.631304 31033 connection.cc:256] RPC call timeout handler was delayed by -1.1883648179996271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1947s for a response
I20260709 12:25:05.630596 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:05.525157  2141 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.524241  1950 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:05.611994  2232 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:05.658025 31065 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:05.611665  2300 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:05.583432  2309 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.617692  2244 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:05.705688  2298 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:25:05.649979  2315 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:05.802868  2000 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.273s	user 0.007s	sys 0.003s
I20260709 12:25:05.627178  2241 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.771586  2141 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:05.777213  1995 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.447s	user 0.004s	sys 0.004s
I20260709 12:25:05.778478  2174 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:05.779847  2301 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.245s	user 0.000s	sys 0.000s
W20260709 12:25:05.780419  2269 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.000s
W20260709 12:25:05.781268  2305 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:05.781922  2187 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:05.630174  2287 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:05.778208  2047 consensus_meta.cc:287] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.609s	user 0.005s	sys 0.000s
W20260709 12:25:05.778012  2017 consensus_meta.cc:287] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.660s	user 0.004s	sys 0.004s
W20260709 12:25:05.803378 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 171060us
I20260709 12:25:05.686975  2300 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.637385  2254 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:05.805534 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.806785  2179 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:05.621876  1965 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.684579  1950 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.849792 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:05.814313 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 183798us
I20260709 12:25:05.852221 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:05.626545  2294 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.822490  1358 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:05.806406  2038 raft_consensus.cc:2764] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:05.611637  2277 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.648757  2265 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.807255 31059 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:05.823975  2304 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:25:05.736203  2309 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.840086  2017 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:05.837052  2300 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:05.847298  2254 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:05.917908 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 65749.4us
I20260709 12:25:05.809572  2000 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.821424  1995 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.802999  2315 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:05.926003  2138 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.321s	user 0.005s	sys 0.012s
W20260709 12:25:05.925802  2279 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.841s	user 0.011s	sys 0.000s
I20260709 12:25:05.824736  2269 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:05.837741  2047 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.918911  2141 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:05.996618 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:05.850715  2174 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
W20260709 12:25:05.978389  1474 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.672s	user 0.008s	sys 0.004s
I20260709 12:25:05.999301 31694 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:05.943204 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1849) took 1563 ms. Trace:
I20260709 12:25:05.858325  2312 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
W20260709 12:25:05.983841  2071 consensus_meta.cc:287] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.697s	user 0.010s	sys 0.000s
I20260709 12:25:06.019392 31137 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:05.977566  2179 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:06.039418 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1845) took 1681 ms. Trace:
W20260709 12:25:06.028537 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 562104us
I20260709 12:25:06.030645 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:06.042611 31063 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:06.030841 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 181110us
I20260709 12:25:06.039898 31135 rpcz_store.cc:276] 0709 12:25:04.357970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.922879 (+564909us) service_pool.cc:224] Handling call
0709 12:25:05.017143 (+ 94264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:05.062239 (+ 45096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.062276 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:05.062739 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.402581 (+339842us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:05.402641 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.402743 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.039175 (+636432us) raft_consensus.cc:1685] finished
0709 12:25:06.039201 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.039351 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44654,"raft.queue_time_us":50774,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":23434,"thread_start_us":339337,"threads_started":1,"wal-append.queue_time_us":398967}
I20260709 12:25:06.028690 31140 rpcz_store.cc:276] 0709 12:25:04.380088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.153560 (+773472us) service_pool.cc:224] Handling call
0709 12:25:05.153972 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:05.154257 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.154302 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:05.154922 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.211460 (+ 56538us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:05.211504 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.211570 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.926815 (+715245us) raft_consensus.cc:1685] finished
0709 12:25:05.926842 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.943146 (+ 16304us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":56124,"threads_started":1,"wal-append.queue_time_us":292830}
I20260709 12:25:06.029043  2297 consensus_queue.cc:1243] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:05.998750  2076 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:06.047880 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51379.5us
W20260709 12:25:06.046775  2310 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 12:25:06.042636  2071 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:06.050449  2320 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:25:06.019539  2146 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:06.053574 31033 leader_election.cc:400] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
I20260709 12:25:06.054244 31033 leader_election.cc:403] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:06.048413 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:06.055106 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:06.056414 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:06.056501 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:06.057758  2205 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:06.058409  2265 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.073413  2047 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:06.074520  2054 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 2.044s	user 0.008s	sys 0.004s
I20260709 12:25:06.074913  2054 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:06.047290  2038 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:06.077883  2269 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:05.987932  2319 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
W20260709 12:25:06.078667  2317 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:25:06.074397  2314 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:25:06.079453  2175 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.080286 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:06.060427  2294 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.062496  2309 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.063992  1358 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:06.081768  2248 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:06.082438  1950 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.082489  1358 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:06.083086  1358 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:06.083580  2287 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:06.084146  2275 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.307s	user 0.009s	sys 0.006s
I20260709 12:25:06.083935  1358 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.085305  1358 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:06.064291 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:06.051142  2071 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.089706  2244 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.064899  2254 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:06.065713  2211 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:06.090552  2211 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:06.090974  2260 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.092545  2211 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.066351 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.067054  2315 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.092973  2260 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:06.093451  2218 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.775s	user 0.008s	sys 0.000s
I20260709 12:25:06.094393 31693 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:06.094689  2000 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:06.096199  2252 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.000s
I20260709 12:25:06.096588  2252 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:06.097762  2236 consensus_meta.cc:287] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.602s	user 0.015s	sys 0.000s
I20260709 12:25:06.058504 31063 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:06.073208  2321 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:25:06.059051  2080 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.081422 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:06.059094  2312 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
I20260709 12:25:06.100183 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:06.101315  2190 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.545s	user 0.005s	sys 0.001s
I20260709 12:25:06.000185  2279 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:05.998111  2138 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:06.102151 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 45748.3us
I20260709 12:25:06.067536 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.103569 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:06.104753 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.139072 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:06.106072 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.116297 31067 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:06.097806  2254 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:06.119376  2284 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.971s	user 0.016s	sys 0.010s
W20260709 12:25:06.128691  2291 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.976s	user 0.000s	sys 0.011s
I20260709 12:25:06.131108 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 863) took 1450 ms. Trace:
I20260709 12:25:06.143592 31694 rpcz_store.cc:276] 0709 12:25:04.680947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.681151 (+   204us) service_pool.cc:224] Handling call
0709 12:25:06.130903 (+1449752us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540044a054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:06.131039 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2670499712}
I20260709 12:25:06.144106 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.144748 30992 raft_consensus.cc:2376] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9. Re-sending same reply.
I20260709 12:25:06.036648  1474 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.139344 31063 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:06.106868 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.146940 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:06.139262 30986 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
W20260709 12:25:06.150867 31691 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.000s
I20260709 12:25:06.151364 31691 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:06.137360  2300 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:06.146601 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.141417 30985 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:06.093863  2218 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.154940 30991 raft_consensus.cc:2410] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:06.155306 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1852) took 1687 ms. Trace:
I20260709 12:25:06.117530  2265 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:06.140605  1990 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:06.069020  2017 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:25:06.155845 31144 rpcz_store.cc:276] 0709 12:25:04.467990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.211764 (+743774us) service_pool.cc:224] Handling call
0709 12:25:05.212039 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.212206 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:05.419718 (+207512us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.121464 (+701746us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b18002e8ca0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:25:06.154986 (+ 33522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:06.155038 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:06.155079 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.155139 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.155242 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":206917,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":3903,"raft.queue_time_us":71,"spinlock_wait_cycles":11388672,"thread_start_us":33238,"threads_started":1}
I20260709 12:25:06.158399 31068 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:25:06.159840 31033 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
W20260709 12:25:06.156618 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 87756.3us
I20260709 12:25:06.163858 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:06.165170 31691 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:06.172972 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.751s (SENT) [suppressed 27 similar messages]
I20260709 12:25:06.176811 31694 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:06.176788 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:06.178757 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:06.117244  2236 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:06.176914 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:06.185312  2283 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.002s	sys 0.009s
W20260709 12:25:06.185997  2283 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.002s	sys 0.009s
I20260709 12:25:06.138348  2072 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:06.188529 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:06.191350 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:06.162557  2312 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:06.192233  1474 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.192418 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:06.193104 31690 raft_consensus.cc:2368] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:25:06.193869 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:06.192955 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:06.195320 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.196435  1531 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.199661  2236 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.201105  2279 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:06.116796  2248 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.160235  1990 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:06.173913  2080 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:06.142781  2190 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.202980  2302 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:06.203339 31067 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:06.181730 31691 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:06.355134 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.219594 31696 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:06.466372 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.454387 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.227345 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:06.215238 30991 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:06.463284 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.466542 30989 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:25:06.220551 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 47620.1us
I20260709 12:25:06.453073 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:06.342288 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 98785.8us
I20260709 12:25:06.498354 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.467388 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:06.504377 31138 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:06.503441 30989 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 614) took 2249 ms (client timeout 1932 ms). Trace:
W20260709 12:25:06.504948 30989 rpcz_store.cc:269] 0709 12:25:04.253632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.262031 (+  8399us) service_pool.cc:224] Handling call
0709 12:25:06.354928 (+2092897us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:25:06.503366 (+148438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3002624}
I20260709 12:25:06.505389 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.505924 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.290663  2300 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.214265  2299 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:06.503085  1965 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:06.509081 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:06.536798 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:06.506194 30986 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
W20260709 12:25:06.510828  2293 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.219s	user 0.013s	sys 0.010s
I20260709 12:25:06.513453  2196 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:06.514671  2218 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.516278  2074 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.516548  2190 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.516752 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:06.517071 30983 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.517444 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:06.274396  1995 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:06.500842 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:06.518678  2248 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:06.520810  2253 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 2.240s	user 0.003s	sys 0.005s
I20260709 12:25:06.522300 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:06.520983 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:06.275012  2259 raft_consensus.cc:993] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:06.530143 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.531528 31689 raft_consensus.cc:2368] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:25:06.532707 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.225244  2312 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:06.532171  2299 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:06.616003 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.227633  2138 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:06.570680 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 381824us
I20260709 12:25:06.571907 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:06.443873  2192 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:06.541594 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:06.640743 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 118300us
I20260709 12:25:06.595580 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.462085  2302 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:06.357342  2277 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:06.598829 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 72018us
I20260709 12:25:06.646364 30983 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:06.468195  2080 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.518468 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:06.600996 31696 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:06.601599 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.652096 31690 raft_consensus.cc:2368] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12.
I20260709 12:25:06.652160 31144 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
W20260709 12:25:06.606264 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 105494us
I20260709 12:25:06.653486 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:06.610110 31135 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
I20260709 12:25:06.614122 31138 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:06.614125 30989 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:06.537556  2148 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.575323  2196 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:06.522866  1990 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.536698  1965 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:06.575933  2260 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:25:06.618893 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 835) took 1143 ms. Trace:
I20260709 12:25:06.623950 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:06.624653 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:06.625295 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:06.627410 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:06.611245  2074 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:06.631685  1531 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:06.634832  2270 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.479s	user 0.008s	sys 0.019s
I20260709 12:25:06.638402 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:06.704840 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.537912 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:06.643803 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2313) took 1774 ms. Trace:
I20260709 12:25:06.637785  2321 raft_consensus.cc:1081] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:06.707062 31694 raft_consensus.cc:2376] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9. Re-sending same reply.
W20260709 12:25:06.707499 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 135703us
I20260709 12:25:06.644368 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:06.648787 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.707376 31067 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:06.709128 31140 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:06.656697 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.709878 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:06.710364 31135 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:06.641575 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.705148 31141 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.711170 31143 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:25:06.711289 31063 raft_consensus.cc:2410] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:06.710471 31067 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.661000 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:06.704773 31062 rpcz_store.cc:276] 0709 12:25:05.475143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.614087 (+138944us) service_pool.cc:224] Handling call
0709 12:25:05.627916 (+ 13829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.851368 (+223452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.851418 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.851475 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:05.851520 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.618665 (+767145us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54002cddd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:06.618713 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.618838 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513354112}
I20260709 12:25:06.741370 31067 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:06.742326 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:06.742995 31067 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:06.678802  2277 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.743887  2258 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.744392  2258 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:06.746595  2258 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.713199 31144 raft_consensus.cc:2410] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:06.690951 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.706504 31060 rpcz_store.cc:276] 0709 12:25:04.869788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.872925 (+  3137us) service_pool.cc:224] Handling call
0709 12:25:04.873288 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.873720 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:04.873762 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.874326 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.043347 (+169021us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:05.043385 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.043452 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.510315 (+1466863us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:06.643481 (+133166us) raft_consensus.cc:1685] finished
0709 12:25:06.643505 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.643735 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"spinlock_wait_cycles":9538304,"thread_start_us":168710,"threads_started":1,"wal-append.queue_time_us":336223}
I20260709 12:25:06.743690 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:06.750206 31060 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:06.750511 31067 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:06.711787 31061 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:06.716571 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:06.640714  2192 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:06.717777 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.752784 30989 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:06.663218 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:06.718796 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:06.655001  2080 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:06.720683 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:06.724195 31141 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
W20260709 12:25:06.724846 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 80599us
I20260709 12:25:06.726845 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1839) took 2378 ms. Trace:
I20260709 12:25:06.727162  2300 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:06.704201  2244 raft_consensus.cc:2764] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 12:25:06.657743 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:06.732856  1990 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.714649  1531 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:06.719678  1995 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:06.757459 31137 rpcz_store.cc:276] 0709 12:25:04.348740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.590392 (+241652us) service_pool.cc:224] Handling call
0709 12:25:04.646645 (+ 56253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.038070 (+1391425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.726630 (+688560us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400063454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:06.726792 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3152147712}
I20260709 12:25:06.756071 30986 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 12.
I20260709 12:25:06.758574 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:06.760257 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 102618us
I20260709 12:25:06.760505 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:06.756731 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:06.715565  2074 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:06.764706 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 101565us
I20260709 12:25:06.766633  2249 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:06.771258  2249 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.771621  2249 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:06.771835 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.773151 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.773753 30986 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:06.719763  2286 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:06.774452  2286 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.774832  2286 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.774775 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.770905 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:06.776408 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:06.776386 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:06.777884 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:06.705871  2148 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.775621 30986 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:06.746024  2277 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.792538 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:06.770116  2278 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:06.804209 31063 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:06.798998 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:06.804167 31062 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:06.776170  2074 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.769997  2236 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:06.769832  2138 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:06.794865  2312 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:06.729163  2299 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.812110 30989 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:06.799660  2017 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:06.800619  2244 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:06.809672 31062 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:06.799624  2069 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:06.802721 31061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60b6187f32774daba89763099dabeb2a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:25:06.803606  2277 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:06.806736  2265 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:06.823607  2236 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:06.807665  1474 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:06.831683  2339 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:06.819376  2279 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:06.833534 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 833) took 1374 ms. Trace:
W20260709 12:25:06.853961 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.853580 31068 rpcz_store.cc:276] 0709 12:25:05.459228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.462657 (+  3429us) service_pool.cc:224] Handling call
0709 12:25:05.463006 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.160839 (+697833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.833311 (+672472us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54002a2954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:06.833479 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598670592}
I20260709 12:25:06.839352 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:06.840914  2074 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.842712 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 546) took 1409 ms. Trace:
I20260709 12:25:06.790416  2300 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.856321 31693 rpcz_store.cc:276] 0709 12:25:05.433059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.433181 (+   122us) service_pool.cc:224] Handling call
0709 12:25:05.445949 (+ 12768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.094753 (+648804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.842497 (+747744us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54006a09d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:06.842655 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490916352}
I20260709 12:25:06.857872  2300 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.832122  1531 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:06.752460  2196 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.809743  2138 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:06.858722  2074 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:06.856336 30988 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.859565  2138 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.823248  2278 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:06.863041 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.756048  2302 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.869038 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.867237 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.846771  2312 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:06.825820  2314 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:25:06.825524  2321 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:25:06.852867  2017 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:06.887205  2017 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.843389  2265 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:06.890717 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1844) took 2533 ms. Trace:
I20260709 12:25:06.890811  2265 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.891350 31139 rpcz_store.cc:276] 0709 12:25:04.357589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.860203 (+502614us) service_pool.cc:224] Handling call
0709 12:25:04.860568 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.921905 (+ 61337us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:05.087091 (+165186us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b180023e360. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:06.080761 (+993670us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b180023e360. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8a75e5 00007f2eeb8a7568
0709 12:25:06.158753 (+ 77992us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.663894 (+505141us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.705493 (+ 41599us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.705640 (+   147us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:06.705854 (+   214us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.706027 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.706105 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.890444 (+184339us) raft_consensus.cc:1685] finished
0709 12:25:06.890496 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.890649 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81166,"prepare.queue_time_us":118846,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2021,"raft.queue_time_us":61457,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":102065,"spinlock_wait_cycles":18087552,"thread_start_us":505747,"threads_started":2,"wal-append.queue_time_us":522563}
I20260709 12:25:06.867785  1474 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:06.895200  2278 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:06.896039  1474 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:06.899220  2148 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.899932  2148 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:06.852777  2069 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:06.903152  2069 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:06.861655  2299 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:06.906121  2069 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:06.846171  2260 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:25:06.876412  2312 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:06.925706  2312 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.913491  1990 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:25:06.931603 30988 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:06.937814 30988 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:06.808781  2192 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:06.983045 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:06.983053 30988 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:25:06.925020  2314 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:06.982519  2192 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.980017  2260 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:06.986549  2334 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:25:07.001749 30987 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:25:06.986730  2314 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:06.940356  2236 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.000169  2260 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:07.024345 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:07.025610  2252 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:07.025794 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:07.039498 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:07.044397 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:07.045760 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:07.048256 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:07.011240  2080 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.060801 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.063566 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.064247 31139 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:07.066452 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.067188 31139 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:07.074184 31141 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:07.078127 31141 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:25:07.078666 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 307817us
W20260709 12:25:07.079026 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 322356us
I20260709 12:25:07.083385 31137 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:07.084259 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.085199 31144 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:07.071259  2244 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.090912 31061 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:07.093137 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.094264 31061 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:07.095506 31142 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:07.097182 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.098145 31144 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:07.104812 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.110831 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.122129 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:07.121950 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 323009us
I20260709 12:25:07.120771 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:07.127092 31058 raft_consensus.cc:2410] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:25:07.128288 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.130321 31033 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 12.
I20260709 12:25:07.130636 31033 leader_election.cc:403] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:07.130966 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.138716 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.042973  2314 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:07.139202 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 283374us
I20260709 12:25:07.135017 31144 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:07.155993 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.157300 31143 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:25:07.133124 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.138955 31135 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:07.139614 31138 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:07.163167 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.130048 31140 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:07.146452 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.166692 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.168570 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.121326  2279 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.182865  2354 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:07.183670  2355 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.184376  2355 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:07.189014 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.189488 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.199632  2351 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:25:07.200232  2174 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:07.200650  2174 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:07.200508  2312 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.201452  2174 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.202457  2174 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:07.204973 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 611) took 3131 ms. Trace:
W20260709 12:25:07.207916  2349 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:25:07.209270 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1854) took 2722 ms. Trace:
I20260709 12:25:07.209602 31136 rpcz_store.cc:276] 0709 12:25:04.487222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.411556 (+924334us) service_pool.cc:224] Handling call
0709 12:25:05.423895 (+ 12339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:05.424227 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.424255 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.619698 (+1195443us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b2c000e06f8. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:25:06.620112 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.800813 (+180701us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:06.800864 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.800952 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.207391 (+406439us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:07.209066 (+  1675us) raft_consensus.cc:1685] finished
0709 12:25:07.209087 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.209213 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":371,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":10171,"spinlock_wait_cycles":178501376,"thread_start_us":180371,"threads_started":1,"wal-append.queue_time_us":377500}
I20260709 12:25:07.216320  2138 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.217373 31067 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:07.217549  2196 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:07.121760  2244 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.223351  2174 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:07.058286  2260 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.174079 31135 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.129632  2027 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:25:07.205349 30990 rpcz_store.cc:276] 0709 12:25:04.073790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.101683 (+ 27893us) service_pool.cc:224] Handling call
0709 12:25:04.101937 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.078554 (+1976617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.078585 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:06.078632 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:06.078671 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.204729 (+1126058us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b5400224a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:07.204777 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.204895 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3088,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"spinlock_wait_cycles":4545105152}
I20260709 12:25:07.235481 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:07.235733 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 379261us
I20260709 12:25:07.236099 31694 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:07.237934 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:07.235755 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:07.238899 31695 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:07.246676 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:07.247293 31698 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.249531 30988 raft_consensus.cc:2410] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:07.252728  2358 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:07.253145 30990 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:07.253120  2358 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:07.261436 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:07.263376 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.170611  2314 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.265875 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:07.267098 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:07.261418 31694 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:25:07.131551  1531 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:25:07.121577  2249 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.264659 31135 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:07.290623 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.266819  2303 raft_consensus.cc:2764] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:07.293494 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.296080 31144 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:07.298198 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.298278 31142 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:07.289165 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:07.310848 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.321930 31144 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:07.266480  2286 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.329725  2235 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:07.330295  2235 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:07.331156  2235 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.295439  2265 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.339118  2138 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:07.297458  2244 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:07.340092  2265 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:07.340883  2249 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:07.341369 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.338135  2286 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:07.342140 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.342821 31136 raft_consensus.cc:2376] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7. Re-sending same reply.
I20260709 12:25:07.328258  2353 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.343667  2080 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:07.343722  2353 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.346035  2353 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:07.347270 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 598) took 3375 ms (client timeout 2893 ms). Trace:
W20260709 12:25:07.347647 30984 rpcz_store.cc:269] 0709 12:25:03.971475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.978819 (+  7344us) service_pool.cc:224] Handling call
0709 12:25:07.347084 (+3368265us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540022d3d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:07.347206 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3364642688}
I20260709 12:25:07.325361  2360 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.354663 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:07.355368 31694 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.358611  2302 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.362363 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.362648 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.362257  1965 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.363528 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.364002  1965 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:07.364604 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.365100 31137 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:07.365664 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.362991 31136 raft_consensus.cc:2376] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9. Re-sending same reply.
I20260709 12:25:07.365984 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.366623 31137 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.366880 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.367722 31136 raft_consensus.cc:2410] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:25:07.368415 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.368789  2312 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:07.363294 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.374450 31143 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:07.374883 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2016) took 1946 ms. Trace:
I20260709 12:25:07.375288 31064 rpcz_store.cc:276] 0709 12:25:05.428834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.430646 (+  1812us) service_pool.cc:224] Handling call
0709 12:25:05.430952 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:05.623466 (+192514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.374637 (+1751171us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54000c6554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:07.374827 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":440675072}
I20260709 12:25:07.375846  2207 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:07.377256 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.393596 31144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:07.394411 31144 raft_consensus.cc:606] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:07.394904 31144 raft_consensus.cc:614] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:07.389941 31698 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:07.402590 31137 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:25:07.403877 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.329197  2017 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:07.403254 31061 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.342134  2174 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 12.
I20260709 12:25:07.411339 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.411439 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:07.414162  2080 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:07.418355  2303 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:07.410588 31694 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:25:07.421212 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 291025us
I20260709 12:25:07.424844 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:07.426159 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:07.427268 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:07.428758 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:07.432397 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:07.429940  2296 raft_consensus.cc:2764] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:07.441042 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:07.407927  2196 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:25:07.430653  2080 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:07.451930  2069 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:07.409956  2192 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:25:07.350340  2360 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.325552  2358 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.435002  2174 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:07.432945  2283 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:07.463506 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.516429 31138 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:07.465301 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.419948  2017 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:07.470201 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 337262us
I20260709 12:25:07.470867 31139 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:25:07.480300 30985 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2048) took 1413 ms (client timeout 1713 ms). Trace:
W20260709 12:25:07.520910 30985 rpcz_store.cc:269] 0709 12:25:06.067135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.067267 (+   132us) service_pool.cc:224] Handling call
0709 12:25:07.480124 (+1412857us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5400312c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:07.480248 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5516160}
I20260709 12:25:07.483573  2296 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:07.484082 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56903.3us
I20260709 12:25:07.484812  2241 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:07.487440  2260 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:07.510592 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:07.531154 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:07.532080 31060 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:07.512069 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:07.532807 31067 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:07.532975 31062 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.513594 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:07.517264 31144 raft_consensus.cc:2410] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:07.534360 31068 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:07.458153 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 316865us
I20260709 12:25:07.524842 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.518560 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:07.537544 31139 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:07.526563 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:07.539101 31698 raft_consensus.cc:2410] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:07.527805 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:07.539481 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:07.528981 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:07.540664 31693 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:07.529170 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.538475 31059 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:07.541318 31137 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:07.539713 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.458856  2190 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:25:07.539661 31695 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
W20260709 12:25:07.553198 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.565292 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.541204 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:07.586705 31142 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:25:07.480326  2174 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:07.516827  2278 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.018s	sys 0.013s
W20260709 12:25:07.626814 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 87161us
W20260709 12:25:07.635588 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.648310 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.650641 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.653486 31068 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:07.654650 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.656349 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.609042 31062 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:07.657286 31137 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:07.543634  2074 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.664489 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:07.667268 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:07.668256 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:07.564823  2358 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.660712 31060 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:07.668903 31067 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:07.670033 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:07.670747 31060 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:07.671260 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:07.661504 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:07.672653 31068 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:07.672812 31067 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:07.563277  2283 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:07.677717 31062 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:07.679217 31062 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:07.517426  2080 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:07.660337  2027 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:07.650676  2278 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.018s	sys 0.013s
I20260709 12:25:07.584889  2296 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:07.692670  2296 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.692711  2283 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.686740  2074 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:07.694952  2369 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:25:07.696035  2027 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:07.544361  2241 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:07.689193 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:07.697366 31142 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:07.622802  2174 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.698104  2241 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:07.584687  2249 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 12:25:07.700106 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2320) took 1076 ms. Trace:
I20260709 12:25:07.700680  2241 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:07.700501 31059 rpcz_store.cc:276] 0709 12:25:06.623588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.623694 (+   106us) service_pool.cc:224] Handling call
0709 12:25:07.699879 (+1076185us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54001bb054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:07.700037 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917195520}
W20260709 12:25:07.701033 30983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2047) took 1635 ms (client timeout 1818 ms). Trace:
I20260709 12:25:07.701301 30986 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:07.701445 30983 rpcz_store.cc:269] 0709 12:25:06.065936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.066068 (+   132us) service_pool.cc:224] Handling call
0709 12:25:07.700883 (+1634815us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b540029a4d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:07.700981 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":974761856}
I20260709 12:25:07.702896 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:07.650520  2131 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:07.719615 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:07.702021 31059 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:07.726585 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.738469 31064 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:07.738907 31064 pending_rounds.cc:85] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:25:07.739185 31064 pending_rounds.cc:107] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:07.739086 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.739657 31064 raft_consensus.cc:2889] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:07.740896 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.745791 31059 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:07.749958 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.748723 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.721751  2131 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:07.760203 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.731841  2312 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:07.671492 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:07.773581 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:07.784312 31060 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:07.787652 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.789700 31059 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:25:07.795360 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 483294us
W20260709 12:25:07.800515 30888 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 312) took 35055 ms (client timeout 15000 ms). Trace:
W20260709 12:25:07.803627 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49580.2us
I20260709 12:25:07.804551 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.805444 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:07.806915 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:07.747133  2374 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:07.812372 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:07.815181 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:07.816468 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.767563  2265 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:07.821154 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:07.752408  2377 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.830799 31033 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
I20260709 12:25:07.827066  2244 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:07.904711 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 74092.9us
I20260709 12:25:07.905931 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:07.907147 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:07.831910 30955 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:07.800997 30888 rpcz_store.cc:269] 0709 12:24:32.727966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.728120 (+   154us) service_pool.cc:224] Handling call
0709 12:25:07.783002 (+35054882us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:05.329257 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:05.786352 (+457095us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:05.786413 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:25:05.786432 (+    19us) tablet.cc:662] Decoding operations
0709 12:25:05.787494 (+  1062us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:05.787614 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:05.787640 (+    26us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:25:06.052132 (+264492us) tablet.cc:701] Row locks acquired
0709 12:25:06.071642 (+ 19510us) write_op.cc:260] PREPARE: finished
0709 12:25:06.071825 (+   183us) write_op.cc:270] Start()
0709 12:25:06.071970 (+   145us) write_op.cc:276] Timestamp: P: 1783599906071764 usec, L: 0
0709 12:25:06.071992 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:25:06.072639 (+   647us) log.cc:844] Serialized 4379 byte log entry
0709 12:25:06.186496 (+113857us) op_driver.cc:464] REPLICATION: finished
0709 12:25:06.641048 (+454552us) write_op.cc:301] APPLY: starting
0709 12:25:06.656620 (+ 15572us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:06.752115 (+ 95495us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:06.774555 (+ 22440us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:06.883212 (+108657us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:06.883681 (+   469us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:25:06.883752 (+    71us) write_op.cc:312] APPLY: finished
0709 12:25:06.884276 (+   524us) log.cc:844] Serialized 83 byte log entry
0709 12:25:06.884477 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:06.884607 (+   130us) write_op.cc:454] Released schema lock
0709 12:25:06.884672 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:06.885072 (+   400us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:06.885108 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1548744,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":387416,"apply.run_cpu_time_us":5094,"apply.run_wall_time_us":275406,"num_ops":11,"prepare.queue_time_us":237359,"prepare.run_cpu_time_us":14694,"prepare.run_wall_time_us":900614,"raft.queue_time_us":979,"raft.run_cpu_time_us":16357,"raft.run_wall_time_us":317270,"replication_time_us":114340,"thread_start_us":428533,"threads_started":4,"wal-append.queue_time_us":109471}]]}
I20260709 12:25:07.894896 31064 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:25:07.789461  2368 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
W20260709 12:25:07.910782 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 469781us
I20260709 12:25:07.826819  2069 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:07.800271  1990 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:07.900357 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 79290.9us
I20260709 12:25:07.952709 31064 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:07.953459 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 121637us
I20260709 12:25:07.783212  2312 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:07.890661  2190 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:07.874137  1889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.011s	sys 0.036s
I20260709 12:25:08.015271 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:07.962827 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:07.818432  2236 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:07.883315  2241 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:07.906066  2235 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.904122  2029 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:08.013384  2369 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 12:25:07.949162  2370 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 12:25:08.015455  2303 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:08.058807 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 106377us
W20260709 12:25:08.059291 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44172.1us
I20260709 12:25:07.908062  2265 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:07.953204  2360 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:08.128594 31106 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.000s	sys 0.055s
W20260709 12:25:08.142095 31106 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.000s	sys 0.055s
W20260709 12:25:08.142840 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 606399us
W20260709 12:25:08.129788  2373 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
I20260709 12:25:08.136575 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:08.057077  1889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.011s	sys 0.038s
I20260709 12:25:07.833387  2289 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.240s
I20260709 12:25:08.035355  1990 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:08.150866 31058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.037s	sys 0.024s
I20260709 12:25:08.065934  2241 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:08.067996  2080 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:08.058339  2017 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:07.905320  2131 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:08.162094 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.056442  2069 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:08.063632  2336 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:08.181844 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.142905  2265 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:08.231657  2265 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:08.191651 31058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.037s	sys 0.024s
I20260709 12:25:08.233351  2336 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:25:08.042021  2196 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.009s	sys 0.039s
W20260709 12:25:08.234429  2196 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.009s	sys 0.048s
I20260709 12:25:08.193204  1990 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:08.234869  2248 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:08.193331  2383 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.236284  2377 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:08.227270  2259 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 12:25:08.239243  2376 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.343s	user 0.003s	sys 0.000s
I20260709 12:25:08.234916  2196 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:08.241479 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.241853  2196 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:08.197264  2386 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:25:08.197943  2380 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.307s	user 0.000s	sys 0.000s
I20260709 12:25:08.244683  2196 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.210803  2382 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.247504  2382 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:08.198894  2381 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:25:08.249621 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 849) took 1446 ms. Trace:
W20260709 12:25:08.216843 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.049s	sys 0.089s
I20260709 12:25:08.251001 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 851) took 1090 ms. Trace:
W20260709 12:25:08.251050 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.049s	sys 0.090s
I20260709 12:25:08.250183 31063 rpcz_store.cc:276] 0709 12:25:06.803143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.807089 (+  3946us) service_pool.cc:224] Handling call
0709 12:25:06.807405 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.092032 (+284627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.092060 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.093842 (+  1782us) log.cc:844] Serialized 52 byte log entry
0709 12:25:07.533003 (+439161us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:07.538045 (+  5042us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.564213 (+ 26168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.564291 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.238713 (+674422us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:08.249236 (+ 10523us) raft_consensus.cc:1685] finished
0709 12:25:08.249361 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.249545 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86829,"prepare.queue_time_us":4508,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":104290,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":37570,"spinlock_wait_cycles":18961792,"thread_start_us":649021,"threads_started":2,"wal-append.queue_time_us":628306}
W20260709 12:25:08.251775 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 589614us
I20260709 12:25:08.251438 31065 rpcz_store.cc:276] 0709 12:25:07.160276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.160825 (+   549us) service_pool.cc:224] Handling call
0709 12:25:07.162848 (+  2023us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.163208 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.163249 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:07.166453 (+  3204us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.662341 (+495888us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:07.662384 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.662449 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.250790 (+588341us) raft_consensus.cc:1685] finished
0709 12:25:08.250811 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.250938 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3480,"raft.queue_time_us":107896,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":10149,"thread_start_us":377145,"threads_started":1,"wal-append.queue_time_us":683537}
I20260709 12:25:08.129220  2235 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.231388  2080 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:08.232327  2069 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:08.236279  2383 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:08.256942 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.259182 31061 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:08.259859 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:38956 (request call id 847) took 1461 ms. Trace:
I20260709 12:25:08.260246 31061 rpcz_store.cc:276] 0709 12:25:06.802470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.802606 (+   136us) service_pool.cc:224] Handling call
0709 12:25:07.891160 (+1088554us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54002832d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88ba3d 00007f2ef085bcd0 00007f2ee9de0254 00007f2ee9de01e2 00007f2ee9de0111 00007f2ee9de008c 00007f2ee9ddf342 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:08.259040 (+367880us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b4c002b2600. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee9b60a20 00007f2ee9b71a29 00007f2ee9b7213f 00007f2ee9b362ab 00007f2ee9b63f3c 00007f2ee9b643fe 00007f2ee9df188b 00007f2eeb7f0495 00007f2eeb84130f 00007f2eeb88ba55 00007f2ef085bcd0
0709 12:25:08.259735 (+   695us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617188992}
I20260709 12:25:08.264853  2392 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.265487  2392 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.278179  2190 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:08.278672  2190 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:08.279680  2190 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.280761  2312 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:08.282120  2174 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.000s
I20260709 12:25:08.282542  2174 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.283145  2241 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:08.289397  2235 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:08.291263  2383 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:08.225514  2244 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.293036  2244 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.295553  2244 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:08.299171  2382 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:08.300058  2131 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:08.307528  2260 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.000s
I20260709 12:25:08.313951  2260 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.314528  2260 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.316486  2260 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:08.322011  2392 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:08.284739  2174 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:08.333853  2358 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.000s
I20260709 12:25:08.333680  1990 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.350001 31065 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:08.350597 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:08.351657 31064 raft_consensus.cc:2410] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:25:08.352375  2074 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:08.352900  2074 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.353293  2074 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.312739  2241 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:08.355197 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:08.355758 31065 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:08.356766  2241 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:08.049226  2276 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 12:25:08.370829  2278 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.374788  2278 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.375560  2074 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:08.376745 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:08.377305  2278 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:08.382432 31067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.007s	sys 0.015s
W20260709 12:25:08.382942 31067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.007s	sys 0.015s
I20260709 12:25:08.383596 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:08.384166 31062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.008s	sys 0.014s
W20260709 12:25:08.384652 31062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.008s	sys 0.014s
I20260709 12:25:08.384424 31061 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:08.389811  2207 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:08.403510 31068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.018s	sys 0.007s
W20260709 12:25:08.403905 31068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.018s	sys 0.007s
I20260709 12:25:08.419692 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 581) took 1340 ms. Trace:
W20260709 12:25:08.419787 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 275807us
I20260709 12:25:08.419952  2266 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:08.420547 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:08.421469  2266 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:08.422766 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.423565  1474 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:08.424183  1474 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:08.431404 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:08.431433 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.020s	sys 0.052s
W20260709 12:25:08.432036 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.020s	sys 0.052s
W20260709 12:25:08.433640 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 526552us
I20260709 12:25:08.435230 31065 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:08.432044 31067 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
W20260709 12:25:08.437543 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 145275us
I20260709 12:25:08.420203 31141 rpcz_store.cc:276] 0709 12:25:07.079464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.080672 (+  1208us) service_pool.cc:224] Handling call
0709 12:25:07.080982 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.081213 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:07.412072 (+330859us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.883159 (+471087us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.883513 (+   354us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.376410 (+492897us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b1800429580. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:08.419194 (+ 42784us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b1800429580. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:25:08.419347 (+   153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.419399 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:08.419450 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.419521 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.419637 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":521198,"prepare.queue_time_us":579468,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":93,"raft.queue_time_us":539205,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":7238,"spinlock_wait_cycles":97768192,"thread_start_us":785778,"threads_started":2,"wal-append.queue_time_us":781713}
I20260709 12:25:08.421931  2266 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:08.440232 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:08.435935  2372 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.005s	sys 0.030s
I20260709 12:25:08.436991  2027 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
W20260709 12:25:08.442359  2372 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.005s	sys 0.030s
I20260709 12:25:08.435071 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:08.411324  1990 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:08.425050  2375 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:08.454836 31106 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:08.442835  2130 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.459290  2266 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:08.462579  2266 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:08.463490 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 852) took 1295 ms. Trace:
I20260709 12:25:08.464635  1531 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:08.466022 31109 leader_election.cc:400] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:25:08.469817  2379 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:08.470608  2378 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.534s	user 0.000s	sys 0.004s
I20260709 12:25:08.473179 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:08.477084  2027 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:08.477602 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:08.474555  2289 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:08.463840 31058 rpcz_store.cc:276] 0709 12:25:07.167879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.168049 (+   170us) service_pool.cc:224] Handling call
0709 12:25:07.168348 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:07.469353 (+301005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.469400 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:07.515546 (+ 46146us) log.cc:844] Serialized 52 byte log entry
0709 12:25:08.233248 (+717702us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:08.233309 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.233408 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.463264 (+229856us) raft_consensus.cc:1685] finished
0709 12:25:08.463287 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.463426 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132241,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":29076,"thread_start_us":995196,"threads_started":2,"wal-append.queue_time_us":923387}
I20260709 12:25:08.493533 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:08.456736  2249 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.497547 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:08.501706 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:08.523233 31109 leader_election.cc:400] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:25:08.447870  2358 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:25:08.525540 31109 leader_election.cc:400] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12.
I20260709 12:25:08.526072 31109 leader_election.cc:403] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:08.496798  2027 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:08.455832  2265 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.529506 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:08.531852 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:08.527616  2207 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:08.528388  2277 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12.
I20260709 12:25:08.501650  1531 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:08.522150  2355 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:08.481678  2130 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.535475 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:08.554955 30986 raft_consensus.cc:2410] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:08.532306  2027 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.497115  2372 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:08.542356 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 405838us
I20260709 12:25:08.543462 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:08.545603 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:08.558969 30988 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.559338 30983 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:08.546412 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:08.495100  2289 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.547374 31059 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:08.536121  1995 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:08.492125  1474 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:08.574435 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:08.576964 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:08.576970 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:08.577715 30992 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:08.550758  2296 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:08.532902  1990 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.522639  2249 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:08.456185  2360 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:08.564419 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:08.554423  2207 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:08.557857  2027 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:08.578446 31135 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:08.590253 31135 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:08.592201 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:08.593178 30987 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:25:08.578446 31032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.004s	sys 0.015s
I20260709 12:25:08.579074 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:08.595391 31032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.005s	sys 0.015s
W20260709 12:25:08.579218 31664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.022s
I20260709 12:25:08.579752 31140 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:25:08.580443 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:08.595806 31664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.001s	sys 0.022s
I20260709 12:25:08.596099 31137 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
W20260709 12:25:08.596156 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 537619us
I20260709 12:25:08.581071 31143 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:08.581647 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:08.597621 31698 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:08.598361 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:08.582675 31693 raft_consensus.cc:2410] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:25:08.548520 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:08.583673 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:08.555578  2303 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:08.584853 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.590999 31135 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:08.606395 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.606743 31135 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:08.584287 30988 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:08.586583 30983 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:08.586680 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:08.586833  2360 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:08.588099 31695 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:25:08.571903 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 436177us
W20260709 12:25:08.596516 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.600533 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 52109.2us
I20260709 12:25:08.602880 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:08.620390 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:08.581223  2296 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.622417 30985 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:08.608467  2074 raft_consensus.cc:2764] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:08.739246 30992 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:25:08.589841  2372 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.608026  1889 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.590791  1990 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.665648  2296 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:08.738440 31135 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:08.768954 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:08.755678 31139 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:08.769788 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:08.770455 31135 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
W20260709 12:25:08.731768  2387 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:25:08.756989 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 135714us
I20260709 12:25:08.712867  2235 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:08.739880  2277 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:08.774412 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:08.748214  2283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.005s	sys 0.008s
I20260709 12:25:08.630591  2360 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:08.770678 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:08.805706 31140 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:08.787482 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:08.810487 31135 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
W20260709 12:25:08.807878 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 188714us
W20260709 12:25:08.798357 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.793987 31664 connection.cc:256] RPC call timeout handler was delayed by -1.5131424369997148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
I20260709 12:25:08.811868 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:08.755962  2355 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:08.765642  2029 consensus_meta.cc:287] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.635s	user 0.008s	sys 0.004s
I20260709 12:25:08.822175 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:08.822299 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:08.773685  2403 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.824023 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:08.821530 31664 connection.cc:256] RPC call timeout handler was delayed by -1.3790424369999528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21110000000000004s for a response
W20260709 12:25:08.809636  2283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.005s	sys 0.009s
I20260709 12:25:08.826635  2130 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:08.734572  2207 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:08.815125  2074 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:08.821774  2265 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:08.819478  2393 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:08.771159  1889 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:08.842051 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.824240  2241 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:08.811547  2235 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.828624  2029 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:08.831466  2131 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.004s
I20260709 12:25:08.849749  2417 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:25:08.817492  2174 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:25:08.767979  2372 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:08.853605  2397 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 12:25:08.844163  2196 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:08.847240  2393 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:08.847828  2400 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 12:25:08.855234  2265 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:08.867480 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.829478  2403 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.855479  2283 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:25:08.854691  2289 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:08.881525 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 53018.7us
I20260709 12:25:08.854132  2236 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 12:25:08.882184 31664 connection.cc:256] RPC call timeout handler was delayed by -1.0840656659997876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
I20260709 12:25:08.870361  2416 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:08.890237 31664 connection.cc:256] RPC call timeout handler was delayed by -1.0282656659996974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 12:25:08.861428  2392 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:25:08.870730  2249 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:08.877285  2396 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:08.899690 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:08.895457 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:08.878659  2417 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.900519 31691 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:08.897822 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:08.900981 31698 raft_consensus.cc:2410] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:25:08.901110 31696 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:08.891981  2416 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:08.902959 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:08.904430 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:08.904570 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:08.907858 31696 raft_consensus.cc:2410] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:08.904749 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:08.904858 31695 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:25:08.908830 31695 pending_rounds.cc:85] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 7
I20260709 12:25:08.909135 31695 pending_rounds.cc:107] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:25:08.909422 31695 raft_consensus.cc:2889] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:08.905761 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:08.914496 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:08.901667 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:08.915146 31697 raft_consensus.cc:2368] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:25:08.915469 31698 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:08.917475 31696 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:08.917682 31691 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:25:08.919070 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:08.924816 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:08.933835 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:08.905714  2408 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:08.935964 31695 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:08.925374  2029 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:08.915678  2417 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.953768  2403 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:08.955225  2241 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:08.955608  2410 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:25:08.956311  2414 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:25:08.956952  2418 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:25:08.957448  2407 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:25:08.906162  2396 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:08.966682  2372 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:08.963785  2411 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:25:08.970140  2417 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:08.970628  2393 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:08.981201  2396 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:08.981906  2393 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.023476 30988 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:09.025803 30986 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:09.029249 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.030927 30989 raft_consensus.cc:2410] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:25:09.031080 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 208876us
I20260709 12:25:09.034384 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.034535 30983 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:09.037864 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:09.038962 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:09.035527 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:09.036254 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:09.036644 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.037247 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:09.043123 30988 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:09.044312 31058 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 12:25:09.059875 30986 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:09.060803 30985 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:08.975312  2296 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.045228 30984 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.106967  2296 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:08.977070  2360 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:08.965754  2377 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:08.965801  2279 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:09.042364 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:09.047371 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 425642us
I20260709 12:25:09.074232  1889 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.074294  2331 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.062669 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:09.079468 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 147811us
I20260709 12:25:08.962949  2241 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:09.083243  2248 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:09.084754  2303 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:09.085440 31694 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:25:09.088709  2190 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.089632  2192 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:08.939772  2074 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:09.090893  1990 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:09.091500  2235 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:09.093300 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.093858 30985 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:09.099272 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.112406 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 70303.3us
W20260709 12:25:09.118624 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.117686 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 55506.4us
I20260709 12:25:09.173738 31694 pending_rounds.cc:85] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 6
I20260709 12:25:09.209745 31059 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:25:09.211723 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.210471 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.193822 30987 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:09.188463  2303 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.123868  2412 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.201072 30989 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:09.258698 30990 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:09.209432 31058 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:09.124573  2314 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.136s
W20260709 12:25:09.260187 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.260078  2403 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:09.260548 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.250932 31694 pending_rounds.cc:107] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:25:09.120299  2406 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:09.260916  2403 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.263634  1474 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.111177  2279 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:09.130529  2296 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.123519  2417 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.193955 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.183221  2192 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:09.271276 30984 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:09.206683 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:09.192977 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:09.279366 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.122648  1889 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.273456  1474 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.275007  2279 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.110702  2377 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:09.267093 31694 raft_consensus.cc:2889] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:09.110183  2360 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.123232  2196 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.300935  2196 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.265718  2406 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.303886 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.304217  2406 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:09.304128  2276 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:25:09.305003 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.305641  2276 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:09.305773 31141 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:09.279219  2412 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.308298 30990 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:09.310088 31144 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 12:25:09.193005  2069 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:09.311456  2069 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.312934  2336 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.313467  2336 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.283401 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.195389  2241 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.123771  2130 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.154s
I20260709 12:25:09.316125  1474 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:09.316514  2130 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:09.317027  2130 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:09.279321 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.294996  1531 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:09.297827 30885 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 5 to 8, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 8 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:09.328739 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 61767.4us
I20260709 12:25:09.300206 31690 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 5.
I20260709 12:25:09.315995 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:09.313890  2069 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.294344  2248 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.309846  2412 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.299252  2192 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.120033  2029 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.166s
I20260709 12:25:09.318069  2130 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.303678  2196 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:09.335083  1889 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:09.347445 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.300953 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.357583 30983 raft_consensus.cc:2410] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:25:09.117149  2331 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.367163 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:09.372103 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 165605us
I20260709 12:25:09.379877 30987 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:09.119136  2416 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.383739 31137 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:09.352713 30885 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 6 to 9, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 9 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.392354 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.402109  2430 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:09.402536 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 86768.7us
I20260709 12:25:09.403033  2248 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.405149 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:09.407487 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.409680 31065 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:09.411160 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.413499 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 190909us
I20260709 12:25:09.424036 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:09.427848  2336 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:09.410800 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.424433 31058 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:09.435151 31064 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:09.435073  2266 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:09.436102  2331 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:09.435932  2266 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:09.436990  2027 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.752s	user 0.014s	sys 0.005s
I20260709 12:25:09.437482  2027 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.437521 30885 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:09.438927 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:09.457791 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 103028us
W20260709 12:25:09.460919 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.400898  2196 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 12:25:09.464056 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 62922.2us
I20260709 12:25:09.417447  1474 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.472999 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 71226.5us
I20260709 12:25:09.378333  2417 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.438155  2266 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:09.451146  2403 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:09.470772 31059 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
W20260709 12:25:09.478843 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.479797  2279 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:09.482853 31142 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:09.484011 30885 catalog_manager.cc:5697] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.438354  2396 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:09.417603  1531 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.531411 31061 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:25:09.533156  2069 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.533674  2069 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.535185  2303 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.535873  2069 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.536229 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:09.537226  2436 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:09.537835  2436 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:09.538964  2433 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.284s	user 0.004s	sys 0.001s
I20260709 12:25:09.539227  2436 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.541169  2436 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.542363  2396 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.544775  2027 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:09.544925  2396 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:09.504168  2196 consensus_queue.cc:1243] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:09.550381  2441 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.551014  2441 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.552770  2403 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.501545  2296 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:09.553303  2441 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:09.553869  2296 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.554322 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.555114 31063 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:25:09.555517  1531 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:09.556200  2296 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.553388  2403 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.556849 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.558182 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:09.558729  2416 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:09.559075  1995 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.978s	user 0.009s	sys 0.009s
I20260709 12:25:09.559384  2192 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.559556  1995 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:09.560693 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 155697us
I20260709 12:25:09.562602 31143 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:09.563129  2425 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:09.563074  1889 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.528800  1474 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:09.571954 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47337.8us
W20260709 12:25:09.576541 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 153193us
I20260709 12:25:09.550040  2130 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.567582 30885 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 6 to 8, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 8 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.586308 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.439832  2074 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.559070  2403 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:09.587175  2276 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:09.605401 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:09.610373  1889 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:09.611721 31063 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:09.612079 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.612699 31060 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:09.612807 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.613619 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.613984 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.614691 31063 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:09.615531 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.615828 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.616261 31063 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:09.616312 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:09.616758 31136 raft_consensus.cc:2410] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:09.617748 31063 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:09.617841 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.619076  2430 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:09.619300  2417 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.598083  2029 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:09.619587  2396 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:09.620301 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:09.621685 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:09.623114 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 87130.3us
I20260709 12:25:09.625661 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:09.626346 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.626976 31067 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:09.627355  2131 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:09.628026 30885 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 8 to 9, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 9 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.630425  2428 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.631165  2428 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.634555 31062 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:09.637247 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.610014 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:09.635811 30885 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 5, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 5 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.601073  2276 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.648115  2276 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:09.651167  2027 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:09.645638  1995 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:25:09.655077 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:09.655411  1995 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:09.655839  1995 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:09.655889  2266 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:09.656929  1995 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.625850  2377 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.596179  1531 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:09.662148 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:09.664078  2377 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.645469  2430 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:09.665791 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:09.666496 31693 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:25:09.666424  2266 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:09.667956 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.668042  2027 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:09.668622  2027 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:09.669093  1995 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.666610 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:09.670845  2417 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:09.671140 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:09.671290  2417 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:09.665534  2430 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.669741  2027 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.671741 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:09.678620 31141 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:09.676699 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:09.683282 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:09.683694 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:09.686795 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 61244us
I20260709 12:25:09.680938 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:09.672084 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:09.683610 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.682855 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:09.674351 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1891) took 1097 ms. Trace:
I20260709 12:25:09.726956 31143 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.729849 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:09.729831 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49042.2us
I20260709 12:25:09.733692 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:09.734587 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 62563.6us
I20260709 12:25:09.734053 30885 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 5 to 6, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 6 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:09.672116  2396 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:09.678376  2241 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:09.671767  2417 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.731849 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:09.681488  2027 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.737607 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:09.687033  2331 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:09.749729 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:09.742028 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61876.1us
I20260709 12:25:09.747500 31060 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:09.740103 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.737323  2279 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.673460  2406 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.710537 31142 rpcz_store.cc:276] 0709 12:25:08.576954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.578520 (+  1566us) service_pool.cc:224] Handling call
0709 12:25:08.590524 (+ 12004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.517536 (+927012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.674001 (+156465us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b540001b654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:09.674273 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2029227904}
I20260709 12:25:09.756472 30885 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 5 to 8, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 8 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.739432  2266 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.762595  2266 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.763084  2266 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.764881 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.765646 31067 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:25:09.767637 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.768314  2425 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.769048  2425 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:09.770453 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 169102us
I20260709 12:25:09.770800  2441 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:09.746474  2303 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:09.738740  2396 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:09.760281 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.773514 31141 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:09.760989 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 89320.8us
I20260709 12:25:09.756148  2192 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.778541 30885 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:09.747797  2428 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.759536  2406 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.792816 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:09.793547 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.793388 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:09.777441  2303 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.778915  2396 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:09.793999  2190 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:09.820659 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.777019  2196 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:25:09.786545  2192 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:09.797569  2441 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:09.825676 31061 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:09.808696  2303 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:09.821843 30885 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 2 to 3, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 3 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:09.840763 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 47468.9us
I20260709 12:25:09.843918  2266 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.799911  1889 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:09.846594  2405 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:09.847069  2236 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:09.850891  2441 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.845732 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:09.854591 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:09.855574 30955 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:09.856359  2428 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:09.857054  2428 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.858562  2283 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:09.858948  2283 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.859273  2283 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.862000  2416 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.862454  2416 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.862879  2416 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.866434  1889 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:09.866932  1889 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.867340  1889 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:09.871012 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:09.827456  2192 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.873895  2443 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:09.874197  2406 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:09.880450  2406 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:09.881000  2425 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:09.881536  2406 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.883263  2406 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.875342  2443 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:09.886782  2443 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:09.817548  2449 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:25:09.837774 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.835315 30885 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:09.878453  2278 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:25:09.914883 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:09.915496 31063 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:09.879882  2428 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:09.923660 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 52755.4us
I20260709 12:25:09.839430  1474 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:09.934928 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:09.935562 30885 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 4 to 7, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 7 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:09.949275 31143 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:09.887806  2443 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.952263 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.829120  1990 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 12:25:09.954233 31060 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:09.955188 31136 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:09.959605 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.969937 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:09.953938  2289 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.973628 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.951124  2266 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:09.951323  2412 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.960366  2436 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:09.898334  2276 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:09.984578 31137 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:09.919203  2336 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:25:09.961336  2130 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:09.990348  2405 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.008347  2405 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:10.010823  2405 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:10.011049  2283 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.990538 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:10.013232  2283 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:10.013477  1889 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:10.013815  2412 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:09.998158 30983 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:25:10.001055 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45425us
I20260709 12:25:10.015357 31692 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:10.015461 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:10.021661 31061 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.016206 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:10.023706 31690 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:25:09.990670 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 237714us
W20260709 12:25:09.990024 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 198250us
I20260709 12:25:10.001883 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:10.002177 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:10.026036 30991 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.003726 31696 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:10.026626 31136 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:09.960811  2196 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:25:10.002236 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.017045  2443 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:10.033735 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:10.033089 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:10.035265 31143 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:10.036408 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:09.995589  2314 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:10.001263  2130 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:10.036412 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.998185  2436 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.037719 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.005949  2279 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:10.003427  2276 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:10.024456  2449 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:10.056641 31137 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:10.038547  2440 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.127s
W20260709 12:25:10.060081 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.052455 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.061678 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:09.994594  2289 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:10.040589  2130 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:10.069509 31692 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 10 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:10.041463  2436 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.057415  2449 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:10.192610 30987 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:25:10.132313 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:10.066407  2279 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:10.343320 31144 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:10.168548 31061 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:10.119315 31142 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:25:10.072816 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 137966us
I20260709 12:25:10.148315 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.344159 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:10.147274 31697 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:25:10.346149 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 318978us
I20260709 12:25:10.125109 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:10.240994 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:10.349375 31141 consensus_meta.cc:287] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.003s
W20260709 12:25:10.308751 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.132069 30885 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 4 to 6, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.275017 30984 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 12:25:10.360731 30984 pending_rounds.cc:85] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 3
I20260709 12:25:10.361479 30984 pending_rounds.cc:107] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:25:10.361869 30984 raft_consensus.cc:2889] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:10.345391 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.358179 31141 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:25:10.351039 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.350404 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.142654 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.357722 30985 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:10.365628 30983 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:10.147488 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.146831 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.367596 31695 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:25:10.366490 30984 pending_rounds.cc:107] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:25:10.371374 30984 raft_consensus.cc:2889] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:10.143157  2451 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:25:10.377455 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:10.377903 31692 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:10.373813 30885 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 7 to 9, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 9 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.375486 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:10.378014 31059 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:10.389788 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.395808  1990 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.396283 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:10.396420  1990 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:10.397768 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 327847us
I20260709 12:25:10.398185 31141 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.398701  1990 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:10.399672 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:10.400857 31697 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:10.401831 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.402701 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:10.403317  2442 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:25:10.403445 31689 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.405570 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:10.406355  2377 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:10.406232 31692 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:10.407411 30885 catalog_manager.cc:5697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.410598  2236 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:10.411834  2236 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:10.411146 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:10.105226  2289 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:10.423904 31689 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:10.344152  2314 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:10.448189 31063 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:10.434165 31141 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:10.452069 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:10.453572 31059 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
W20260709 12:25:10.392504  2396 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.000s
W20260709 12:25:10.444309 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.153597  2416 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:10.443509 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 49576.6us
I20260709 12:25:10.369897  2336 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:11.043344 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:11.087615 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 687992us
W20260709 12:25:10.676129 30988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.043s	sys 0.104s
I20260709 12:25:10.590672 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:11.089300 30988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.074s	sys 0.122s
W20260709 12:25:10.807823 31694 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.915s	user 0.010s	sys 0.000s
I20260709 12:25:10.518280 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:10.639998 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:10.776386 30991 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.740s	user 0.002s	sys 0.005s
W20260709 12:25:10.460402 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 404104us
I20260709 12:25:10.695127 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:11.123008 30991 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:25:10.797600 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.691649 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.113488 30983 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:25:10.676017 31143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.017s	sys 0.023s
W20260709 12:25:11.124109 31143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.023s	sys 0.029s
W20260709 12:25:10.894151 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.008s	sys 0.018s
I20260709 12:25:10.513690 30990 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:25:10.884270  2440 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.125463 30990 pending_rounds.cc:85] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 5
I20260709 12:25:11.125989 30990 pending_rounds.cc:107] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:25:11.125670  2440 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.126343 30990 raft_consensus.cc:2889] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:11.119822 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 641) took 1816 ms. Trace:
W20260709 12:25:10.451128  2452 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
I20260709 12:25:11.121461 31694 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
W20260709 12:25:11.033787 31067 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.003s	sys 0.006s
W20260709 12:25:11.132970 31067 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.005s	sys 0.007s
I20260709 12:25:11.123324 31062 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:11.101505 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.124173 31063 raft_consensus.cc:2410] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:25:10.499991 31061 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
W20260709 12:25:11.125445 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 737015us
W20260709 12:25:10.819450  2303 consensus_meta.cc:287] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.955s	user 0.011s	sys 0.000s
I20260709 12:25:10.522928 31059 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:25:10.521675  2192 consensus_meta.cc:287] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.647s	user 0.007s	sys 0.004s
W20260709 12:25:10.771343  2453 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.714s	user 0.000s	sys 0.001s
I20260709 12:25:10.513778  2248 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:10.781303  2130 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.001s
I20260709 12:25:11.134822 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:10.691365  2406 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:11.148422  2406 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.151187 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:11.154274 30985 raft_consensus.cc:2368] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11.
I20260709 12:25:11.158473  2454 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:11.155364  2130 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.159221  2454 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:10.830937 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:10.637164  2457 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.595s	user 0.001s	sys 0.000s
W20260709 12:25:10.594925  2459 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 12:25:11.164914  2336 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.165786  2130 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:10.729480 31060 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:11.170401 31059 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:25:11.170606  2456 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.063s	user 0.012s	sys 0.005s
W20260709 12:25:11.166082  2466 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.563s	user 0.000s	sys 0.012s
I20260709 12:25:11.172060 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1909) took 1115 ms. Trace:
I20260709 12:25:11.146047  2303 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:10.455928  2472 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.681s
I20260709 12:25:11.087527  2314 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:10.455287  2449 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.161520 30992 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:11.175304 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:10.604797  2027 consensus_meta.cc:287] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.848s	user 0.007s	sys 0.002s
I20260709 12:25:11.176369  2027 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.136914  2074 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.177120  2027 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.150053  2192 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.180653  2192 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.182901  2027 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.158661  2393 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:11.184753  2393 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:11.128806  1995 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.459s	user 0.009s	sys 0.009s
W20260709 12:25:11.138278 31136 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.000s	sys 0.019s
W20260709 12:25:10.601749  2417 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.840s	user 0.005s	sys 0.003s
I20260709 12:25:11.187073  2417 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.189628 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:11.159646  2406 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.160836 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:11.172415 31143 rpcz_store.cc:276] 0709 12:25:10.057139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.057375 (+   236us) service_pool.cc:224] Handling call
0709 12:25:10.057726 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.105829 (+ 48103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.105865 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.106436 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.125032 (+1018596us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:11.125071 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.125134 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.171859 (+ 46725us) raft_consensus.cc:1685] finished
0709 12:25:11.171882 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.172005 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1529,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":137833,"thread_start_us":1018197,"threads_started":1,"wal-append.queue_time_us":1046493}
I20260709 12:25:11.172977  2416 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.178612 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2048) took 1759 ms. Trace:
I20260709 12:25:11.183763  2236 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.183903 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 880) took 1252 ms. Trace:
I20260709 12:25:11.196352 31064 rpcz_store.cc:276] 0709 12:25:09.418985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.436139 (+ 17154us) service_pool.cc:224] Handling call
0709 12:25:09.436478 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.550437 (+113959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.550483 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.551166 (+   683us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.826374 (+275208us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:09.826415 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.826494 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.770582 (+944088us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:11.178299 (+407717us) raft_consensus.cc:1685] finished
0709 12:25:11.178326 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.178534 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88636,"raft.queue_time_us":115610,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":164,"spinlock_wait_cycles":6222208,"thread_start_us":274720,"threads_started":1,"wal-append.queue_time_us":465492}
I20260709 12:25:11.196506 31058 rpcz_store.cc:276] 0709 12:25:09.931659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.931818 (+   159us) service_pool.cc:224] Handling call
0709 12:25:09.932151 (+   333us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:09.977691 (+ 45540us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:10.015976 (+ 38285us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.517377 (+501401us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:10.678635 (+161258us) log.cc:844] Serialized 52 byte log entry
0709 12:25:10.678853 (+   218us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:10.678904 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.678991 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.183637 (+504646us) raft_consensus.cc:1685] finished
0709 12:25:11.183670 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.183849 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150888,"prepare.queue_time_us":61718,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":85878,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":97004,"thread_start_us":425356,"threads_started":1,"wal-append.queue_time_us":1165484}
I20260709 12:25:10.514361  2377 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:11.200810  2405 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:11.201323  2405 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:11.205761  2405 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.153575  2428 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:11.207275  2428 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:10.593392  2460 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
I20260709 12:25:11.214663  2412 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.215603  2393 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.215758  1889 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.215938 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:11.216545 31143 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:11.176329 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:11.239538 30992 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:11.234730  2278 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.007s	sys 0.007s
I20260709 12:25:11.121034  2396 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.151741  2248 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:11.127588 30988 rpcz_store.cc:276] 0709 12:25:09.273565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.393151 (+119586us) service_pool.cc:224] Handling call
0709 12:25:09.412072 (+ 18921us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:09.507797 (+ 95725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.507833 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:09.535903 (+ 28070us) log.cc:844] Serialized 26 byte log entry
0709 12:25:09.814152 (+278249us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:09.928558 (+114406us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.513575 (+585017us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b1800424cc0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a7522 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:10.587564 (+ 73989us) spinlock_profiling.cc:243] Waited 63.8 ms on lock 0x7b2400175f58. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee862d9c4 00007f2ee862bacb 00007f2ee87e6dc9 00007f2ee87e6a55 00007f2ee87e5fb0 00007f2eeb45c488 00007f2eeb45cbe8 00007f2eeb434398 00007f2eeb42f76a 00007f2eeb42ebb6
0709 12:25:11.089741 (+502177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.089855 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.089894 (+    39us) raft_consensus.cc:1685] finished
0709 12:25:11.089909 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.090020 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47997,"prepare.queue_time_us":94584,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":58785,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":22816,"spinlock_wait_cycles":157378944,"thread_start_us":1415317,"threads_started":2,"wal-append.queue_time_us":439728}
I20260709 12:25:11.250411 30989 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:11.235440  2029 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.274752 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:11.286480 30983 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:11.275411 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 143318us
W20260709 12:25:11.279891 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 145172us
I20260709 12:25:11.135281 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2077) took 1131 ms. Trace:
W20260709 12:25:11.198045 31136 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.000s	sys 0.019s
W20260709 12:25:11.280483 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 166124us
I20260709 12:25:11.279217 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:11.236341 30985 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:11.169420  2130 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.287744 30991 rpcz_store.cc:276] 0709 12:25:09.992061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.998237 (+  6176us) service_pool.cc:224] Handling call
0709 12:25:11.123747 (+1125510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:11.290753 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:11.291610 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:11.293936 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:11.294720 31143 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:25:11.283399  2278 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.008s	sys 0.008s
I20260709 12:25:11.296960 30992 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:25:11.177584  2074 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.285593  2393 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:11.345232 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 55991.6us
I20260709 12:25:11.336581  2449 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:11.336840  2463 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.318851 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:11.354348 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52742.4us
W20260709 12:25:11.337991  2436 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.220s	user 0.010s	sys 0.006s
I20260709 12:25:11.292068  2417 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.346189 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:11.440317 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 94228.4us
I20260709 12:25:11.441340 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:11.435995 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 73712.5us
I20260709 12:25:11.448024 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:11.448685 31060 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:11.450295 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.450095 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:11.454193 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.354089  2027 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.450212 31694 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 559) took 1879 ms (client timeout 2241 ms). Trace:
I20260709 12:25:11.361441  2393 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:11.467749 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.299410  2130 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.469436 30992 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:11.458772 31694 rpcz_store.cc:269] 0709 12:25:09.557640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.557850 (+   210us) service_pool.cc:224] Handling call
0709 12:25:11.437122 (+1879272us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54004d7e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:11.437370 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751777536}
I20260709 12:25:11.290555  2029 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:11.438618  1474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.228s	user 0.029s	sys 0.042s
I20260709 12:25:11.461890 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:11.472968 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:11.362782  2425 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:11.321273  2412 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.123s
I20260709 12:25:11.474776 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:11.475247 31143 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:11.360685  2485 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:11.481150 30984 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:11.480330 30985 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:11.454953 31058 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:11.485976 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2081) took 1411 ms. Trace:
I20260709 12:25:11.486933 31063 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:11.486667  2485 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.487916 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:11.487607 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 362852us
I20260709 12:25:11.486377 30989 rpcz_store.cc:276] 0709 12:25:10.074558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.075417 (+   859us) service_pool.cc:224] Handling call
0709 12:25:11.485813 (+1410396us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b54001608d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:11.485923 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077449600}
I20260709 12:25:11.454872  2416 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:11.496919 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.497552 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:11.499414 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:11.499820 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:11.500694 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:11.482549  1474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.266s	user 0.029s	sys 0.044s
I20260709 12:25:11.363519  2436 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.515779 30988 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:11.514518 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:11.519375 31058 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:11.495946  2241 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.748s	user 0.015s	sys 0.006s
I20260709 12:25:11.514681 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.491554  2428 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:11.500270 31109 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.712s (SENT) [suppressed 16 similar messages]
I20260709 12:25:11.516513 31059 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:11.496335  2235 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:25:11.516909  2303 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.479946  2336 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:11.517582 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.517750 31696 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:11.482012  2454 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.493909  2441 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:11.491358  2440 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.488780  1889 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.485211  2469 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.439818  2463 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.522042 31064 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:11.457814  2283 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:25:11.500546 31065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.041s	user 0.006s	sys 0.009s
I20260709 12:25:11.502558  2192 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:11.519994  2278 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 12:25:11.543350  2443 consensus_meta.cc:287] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.500s	user 0.015s	sys 0.004s
I20260709 12:25:11.521277  2412 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:11.492631  2425 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.524804  1995 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.542163 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.514348  2472 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:25:11.502807  2027 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:11.547746 31065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.006s	sys 0.009s
I20260709 12:25:11.496560  2485 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:11.541695  2482 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:25:11.579676 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 883) took 1214 ms. Trace:
I20260709 12:25:11.546101  1889 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:11.550953  2476 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:25:11.531075 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:11.550895  2393 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.552304 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:11.562326 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.548192 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:11.538880  2454 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:11.533403  2377 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:11.548738  2441 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:11.547078  2396 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:11.548484  2436 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.565681  2428 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:11.560810  2074 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.568427  2472 consensus_queue.cc:1243] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:11.637058 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:11.638393 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 346845us
I20260709 12:25:11.644075 31142 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:11.568508  1995 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.547127  2469 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:11.640277  2483 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.354s	user 0.001s	sys 0.000s
W20260709 12:25:11.650741 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 150991us
I20260709 12:25:11.637094  2412 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.662261 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:11.663621 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 174424us
I20260709 12:25:11.674921 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:11.675671  2406 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:11.676322  2406 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:11.676726  2406 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:11.677516  2406 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.679792 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2070) took 1320 ms. Trace:
I20260709 12:25:11.680225 31067 rpcz_store.cc:276] 0709 12:25:10.359125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.397256 (+ 38131us) service_pool.cc:224] Handling call
0709 12:25:10.397577 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.135436 (+737859us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:11.135476 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.136010 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.487376 (+351366us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:11.487412 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.487472 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.679574 (+192102us) raft_consensus.cc:1685] finished
0709 12:25:11.679592 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.679732 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87899,"raft.queue_time_us":116010,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"thread_start_us":910819,"threads_started":2,"wal-append.queue_time_us":404646}
I20260709 12:25:11.681275  2412 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.681941 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2313) took 1234 ms. Trace:
I20260709 12:25:11.682514 30986 rpcz_store.cc:276] 0709 12:25:10.447502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.191655 (+744153us) service_pool.cc:224] Handling call
0709 12:25:11.192056 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.192416 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.192442 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.192986 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.494837 (+301851us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:11.494872 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.494961 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.681716 (+186755us) raft_consensus.cc:1685] finished
0709 12:25:11.681735 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.681871 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":301491,"threads_started":1,"wal-append.queue_time_us":424115}
W20260709 12:25:11.703717 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.715013 31142 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:25:11.711225 31696 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 894) took 1564 ms (client timeout 1557 ms). Trace:
I20260709 12:25:11.630296  2454 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.652339  2396 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:11.719444 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 188427us
I20260709 12:25:11.684489 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:11.651770  2428 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:11.719295 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:11.660871  2405 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:11.717245 31696 rpcz_store.cc:269] 0709 12:25:10.146585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.150662 (+  4077us) service_pool.cc:224] Handling call
0709 12:25:10.639801 (+489139us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:25:11.710972 (+1071171us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54005e7754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:11.711140 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919166080}
I20260709 12:25:11.709111  2130 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.715767 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:11.731879 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2318) took 1044 ms. Trace:
W20260709 12:25:11.727396 31062 consensus_meta.cc:287] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.008s
I20260709 12:25:11.727620 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:11.729130 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 63307us
I20260709 12:25:11.721571 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.731709 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1912) took 1508 ms. Trace:
I20260709 12:25:11.735749 31689 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:11.736017 31059 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:11.735383 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1911) took 1610 ms. Trace:
I20260709 12:25:11.736689 31062 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:11.738477 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:11.729983  2484 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 12:25:11.740898 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:11.725682  2396 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.741736 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:11.740833 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:11.744453 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2355) took 1216 ms. Trace:
I20260709 12:25:11.745015 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.737962 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:11.733711 30987 rpcz_store.cc:276] 0709 12:25:10.687421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.217702 (+530281us) service_pool.cc:224] Handling call
0709 12:25:11.218047 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.218290 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:11.458734 (+240444us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.731608 (+272874us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:11.731668 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:11.731708 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.731765 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.731838 (+    73us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":328403,"prepare.run_cpu_time_us":314,"prepare.run_wall_time_us":2231,"raft.queue_time_us":81,"thread_start_us":512621,"threads_started":2}
I20260709 12:25:11.741521 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:11.755993 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1910) took 1684 ms. Trace:
I20260709 12:25:11.731096  2029 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.754859 30986 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:11.733845  2236 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:11.757249 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:11.751139  2396 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:11.748322 31062 rpcz_store.cc:276] 0709 12:25:10.520455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.694419 (+173964us) service_pool.cc:224] Handling call
0709 12:25:11.737162 (+1042743us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:11.760012 31661 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:11.744854  2509 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:11.790537  2509 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.792820  2509 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:11.797458 31689 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:11.799155 31058 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:11.756407 31137 rpcz_store.cc:276] 0709 12:25:10.071653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.072166 (+   513us) service_pool.cc:224] Handling call
0709 12:25:10.131961 (+ 59795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.223444 (+ 91483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.223487 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.368680 (+145193us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.683840 (+315160us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:10.683882 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.683980 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.755811 (+1071831us) raft_consensus.cc:1685] finished
0709 12:25:11.755829 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.755942 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":284378,"threads_started":1,"wal-append.queue_time_us":850452}
I20260709 12:25:11.748059 31695 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:11.754858 31067 raft_consensus.cc:2410] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:11.738071  2504 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:11.747987  2412 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.738971  2487 raft_consensus.cc:2764] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 12:25:11.786376 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2064) took 1747 ms. Trace:
W20260709 12:25:11.787122  2497 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:25:11.747064  2481 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:25:11.803805 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:11.736915 31136 rpcz_store.cc:276] 0709 12:25:10.125531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.125670 (+   139us) service_pool.cc:224] Handling call
0709 12:25:10.344205 (+218535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.344440 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:11.288524 (+944084us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.735088 (+446564us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:11.735135 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:11.735183 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.735246 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.735341 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":735115,"prepare.run_cpu_time_us":3658,"prepare.run_wall_time_us":16799,"raft.queue_time_us":42670,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":1331724,"threads_started":3}
I20260709 12:25:11.746968  2500 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:11.802875 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58479.5us
W20260709 12:25:11.803360 31109 leader_election.cc:336] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.134s (SENT)
W20260709 12:25:11.800343  2496 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:25:11.737993  2502 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:25:11.817719 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:11.734122  2377 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.733989 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:11.854720 31142 raft_consensus.cc:606] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:11.855204 31142 raft_consensus.cc:614] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:11.839628  2412 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.804862  2436 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:11.810938 31068 rpcz_store.cc:276] 0709 12:25:09.976339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.976815 (+   476us) service_pool.cc:224] Handling call
0709 12:25:09.993908 (+ 17093us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.029354 (+ 35446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.029385 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.038840 (+  9455us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.441309 (+402469us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:10.441351 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.441442 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.723401 (+1281959us) raft_consensus.cc:1685] finished
0709 12:25:11.723422 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.723541 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9317,"raft.queue_time_us":4197,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":20024,"thread_start_us":371137,"threads_started":1,"wal-append.queue_time_us":1231307}
I20260709 12:25:11.806210  2504 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:11.849026 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:11.731341  2441 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:11.817615  2074 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:11.801723  1474 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 12:25:11.864786  2489 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.234s	user 0.001s	sys 0.001s
I20260709 12:25:11.865496 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:11.879921 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:11.884265 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:11.850663  2130 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:11.869908  2469 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.844028  1889 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.821202  2463 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:11.870544 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2314) took 1352 ms. Trace:
W20260709 12:25:11.923967 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.884284 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.735971 31138 rpcz_store.cc:276] 0709 12:25:10.223783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.256215 (+ 32432us) service_pool.cc:224] Handling call
0709 12:25:10.334198 (+ 77983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.450661 (+116463us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:11.141546 (+690885us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.727040 (+585494us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:11.727087 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:11.731467 (+  4380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.731551 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.731656 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":694313,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":7143,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":3709,"thread_start_us":967058,"threads_started":2}
I20260709 12:25:11.921092 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:11.925571 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.918644 31136 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:11.926544 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:11.928118 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:11.923166 30990 rpcz_store.cc:276] 0709 12:25:10.517746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.207652 (+689906us) service_pool.cc:224] Handling call
0709 12:25:11.208083 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:11.208525 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.208565 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:11.209248 (+   683us) log.cc:844] Serialized 52 byte log entry
0709 12:25:11.526354 (+317106us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:11.638669 (+112315us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.641690 (+  3021us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.643177 (+  1487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.643275 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.870300 (+227025us) raft_consensus.cc:1685] finished
0709 12:25:11.870323 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.870485 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":739,"prepare.queue_time_us":37142,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":4674,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":7409,"spinlock_wait_cycles":30573568,"thread_start_us":412660,"threads_started":2,"wal-append.queue_time_us":338335}
I20260709 12:25:11.928707 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 897) took 1540 ms. Trace:
I20260709 12:25:11.927529  2472 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:11.932705  2472 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:11.933038 31689 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:11.929068 31693 rpcz_store.cc:276] 0709 12:25:10.379492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.379630 (+   138us) service_pool.cc:224] Handling call
0709 12:25:10.403044 (+ 23414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.403508 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.403535 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.404059 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.695605 (+291546us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:10.695646 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.695710 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:11.919810 (+1224100us) raft_consensus.cc:1685] finished
0709 12:25:11.919832 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.919972 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1245,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":291176,"threads_started":1,"wal-append.queue_time_us":915359}
I20260709 12:25:11.934165 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:11.934386 31064 raft_consensus.cc:2410] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:25:11.863003  2436 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.875948  2509 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.934871 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:11.954762 31068 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:11.876258  2027 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.955641 31694 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
W20260709 12:25:11.959998  2479 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:25:11.961243  2492 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.961555  2495 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.214s
I20260709 12:25:11.962365  2513 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:11.962796  2512 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.933080  2472 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:11.930496  2074 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:11.963967  2336 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.964437  1995 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:11.965054  2192 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.034904 31144 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:11.966099  2493 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 12:25:11.966643  2248 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.968139 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:11.970853 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:11.842315  2500 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:11.964028  2503 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:25:11.876346  2412 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:11.998989 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:12.050346 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.989001  2241 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:11.938757  2436 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.934469  2507 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:11.875711  2504 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.043148  2500 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.935789  2425 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:11.967756  2416 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 12:25:12.045123 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 77064.6us
W20260709 12:25:12.045925 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.050642 31138 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:11.999981 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 196310us
I20260709 12:25:12.057585 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:11.879343  2428 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:12.051419 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 255964us
I20260709 12:25:12.054169 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:12.021999 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.039963  2481 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:11.976691  2509 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.076395 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:12.077045 31140 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:12.050181  2393 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:12.053443  2507 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.063758  2314 consensus_meta.cc:287] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.889s	user 0.014s	sys 0.010s
I20260709 12:25:12.079051 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.075557 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:12.053191  2130 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:11.978122  2492 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.063658  2454 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:12.044879  2192 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.071646  2248 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.107513 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:12.108387 31144 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:12.108294  2528 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:12.106009 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:12.044677  2336 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.107625  2514 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:12.109311 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:12.079075  2500 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.108894  2528 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:12.119822 31109 leader_election.cc:336] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.173s (SENT)
I20260709 12:25:12.079425  2393 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.000562  2513 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.121106 31144 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:12.104511 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:12.063757  2241 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.076308  2469 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:25:12.076478  2196 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:25:12.126305 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:12.126258 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.125797 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:12.140599 31139 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:12.129662 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:12.105425  2303 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:12.119472  2511 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:12.104138  2507 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.137938  2443 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:12.048543  2512 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.150871 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:12.130373  2393 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.139000  2528 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:12.123560  2440 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:25:12.110837  2314 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.121132  2074 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.131726  2500 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.145968  1995 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:12.144330  2303 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.214591 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.480849 31138 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
W20260709 12:25:12.479563 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 369773us
I20260709 12:25:12.213335 31139 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:12.180164 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.151063  2495 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.245513 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 115984us
I20260709 12:25:12.245360 31136 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:12.245937  2510 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.271s	user 0.001s	sys 0.001s
I20260709 12:25:12.158573  1474 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:12.489235 31059 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:25:12.227208  2518 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 12:25:12.248687  2449 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.890s	user 0.012s	sys 0.000s
I20260709 12:25:12.229244 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:12.478308  2513 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:12.494982 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.010s
W20260709 12:25:12.503542 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.006s	sys 0.010s
I20260709 12:25:12.227543  2192 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.230449  2443 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.525795 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:12.227458  2523 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:12.231959  2509 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:12.482496  2303 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.495361 31138 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:25:12.506395  2454 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:12.507264  2428 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:12.489296  2514 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.521127  2405 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.787s	user 0.008s	sys 0.003s
I20260709 12:25:12.555893 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.556484 31059 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:12.499600 31135 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:12.231000  2241 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.167302  2130 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:12.529343 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 427133us
I20260709 12:25:12.489611  1995 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:12.553647 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:12.555330 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 606) took 1039 ms. Trace:
W20260709 12:25:12.564319 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.524205 31068 consensus_meta.cc:287] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.000s
I20260709 12:25:12.567498 31068 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:12.567499  2412 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.493014  2495 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.531003  2512 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:12.538517  2520 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:12.565577 31141 rpcz_store.cc:276] 0709 12:25:11.515483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.540684 (+ 25201us) service_pool.cc:224] Handling call
0709 12:25:12.034440 (+493756us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:25:12.034634 (+   194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.063599 (+ 28965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.063634 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.068124 (+  4490us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.175964 (+107840us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:12.176231 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.176458 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.176529 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.542634 (+366105us) raft_consensus.cc:1685] finished
0709 12:25:12.555117 (+ 12483us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.555255 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9125,"prepare.queue_time_us":831,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":9412,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":10910,"spinlock_wait_cycles":7690496,"thread_start_us":55572,"threads_started":1,"wal-append.queue_time_us":420559}
I20260709 12:25:12.577873 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.577854 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.557132  2514 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.582542  2520 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:12.583139  2520 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:12.586411  2504 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.003s
I20260709 12:25:12.587005  2504 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.589600  2504 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.580124 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:12.602777 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 608) took 1082 ms. Trace:
I20260709 12:25:12.603284 31143 rpcz_store.cc:276] 0709 12:25:11.520532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.670878 (+150346us) service_pool.cc:224] Handling call
0709 12:25:11.671278 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.682831 (+ 11553us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:11.682861 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.683318 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.880535 (+197217us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:11.880587 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.880674 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.602507 (+721833us) raft_consensus.cc:1685] finished
0709 12:25:12.602533 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.602705 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10704,"raft.queue_time_us":10863,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":196694,"threads_started":1,"wal-append.queue_time_us":293130}
I20260709 12:25:12.605741 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.607968  2500 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.609485 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:12.609899 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.611744 31061 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:12.618777 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45918.7us
W20260709 12:25:12.622423 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.568912  2436 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 12:25:12.636122 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:12.533064  2443 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.574132  2241 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.640093 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 135196us
I20260709 12:25:12.637567  2289 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.618484  2449 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:12.704779 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 123216us
I20260709 12:25:12.625161  2283 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.573812  2539 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:12.707901 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 71852us
I20260709 12:25:12.704934 31106 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:12.562852  2428 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.562736  2454 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.614974  1995 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.652750  2495 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.705276  2504 raft_consensus.cc:2764] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:12.717645 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.639148  2523 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.711246  2547 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.637914  2405 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:12.639932  2314 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.646821  2440 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:25:12.707319  2241 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:12.711725  2531 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
W20260709 12:25:12.707104 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 222394us
I20260709 12:25:12.723407  2406 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:12.726183  2512 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:12.799049 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:12.723407  2396 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.918s	user 0.006s	sys 0.006s
I20260709 12:25:12.639842  2436 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.790496 31059 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:12.817629 31689 pending_rounds.cc:85] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 4
I20260709 12:25:12.821197 31689 pending_rounds.cc:107] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:25:12.821450 31689 raft_consensus.cc:2889] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:12.723830  2283 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.741595  2492 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:12.711979  2289 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.840478  2289 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.829869  2406 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:12.841231  2514 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:12.843077  2406 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.843506  2428 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:11.602869 31065 rpcz_store.cc:276] 0709 12:25:10.365383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.457833 (+ 92450us) service_pool.cc:224] Handling call
0709 12:25:10.458242 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.458489 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:10.458840 (+   351us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.579353 (+1120513us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:11.579401 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:11.579456 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.579516 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:11.579625 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15536,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":23736,"thread_start_us":1119763,"threads_started":1}
I20260709 12:25:12.847124 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.847429 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 901) took 1121 ms. Trace:
I20260709 12:25:12.847739 31065 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:12.847975 31698 rpcz_store.cc:276] 0709 12:25:11.726083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.726371 (+   288us) service_pool.cc:224] Handling call
0709 12:25:11.746016 (+ 19645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.756430 (+ 10414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.756470 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.794963 (+ 38493us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.049449 (+254486us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:12.049488 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.049547 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.847171 (+797624us) raft_consensus.cc:1685] finished
0709 12:25:12.847199 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.847359 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":439,"spinlock_wait_cycles":46219136,"thread_start_us":239178,"threads_started":1,"wal-append.queue_time_us":369531}
I20260709 12:25:12.780992  2547 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:12.850528 31109 leader_election.cc:336] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.311s (SENT)
W20260709 12:25:12.851622 31109 leader_election.cc:336] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.422s (SENT)
I20260709 12:25:12.852006 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:12.852734  2547 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.853642  2454 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:12.830837 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.865902 31058 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:12.782282  2523 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.869277  2523 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:12.832552  2536 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:25:12.832473 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:12.832895  2472 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.784s	user 0.006s	sys 0.003s
I20260709 12:25:13.142659  2472 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:13.143553 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 311117us
W20260709 12:25:12.834968  2393 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.000s
I20260709 12:25:13.144647 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:13.144717  2393 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.835384  2241 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:12.836588  2443 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:13.146425  2547 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:12.837958  2485 consensus_meta.cc:287] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.222s	user 0.012s	sys 0.000s
I20260709 12:25:13.146881  2547 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.147328  2547 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.147055  2485 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:12.877482 31065 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:13.385579 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2367) took 1230 ms. Trace:
I20260709 12:25:13.385953 31065 rpcz_store.cc:276] 0709 12:25:12.155580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.876966 (+721386us) service_pool.cc:224] Handling call
0709 12:25:12.877272 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.385295 (+508023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.385506 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.386926 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.387416 31065 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:13.388099 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.388913 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.389421 31065 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:12.887624 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:12.835647 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:12.818275  2512 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:13.391912 31141 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:13.392441  2512 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:13.392710 31068 raft_consensus.cc:2410] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:13.393137 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:13.089607  2507 consensus_meta.cc:287] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.933s	user 0.010s	sys 0.000s
I20260709 12:25:12.726954  2539 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:13.394459  2507 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.395923  2539 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:13.130823  2303 consensus_meta.cc:287] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.001s
I20260709 12:25:13.396548  2507 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:13.131217  2520 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.003s
I20260709 12:25:13.396631  2303 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:13.131923  2336 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.133972  2436 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:12.827102  2396 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:13.398936  2303 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.399353  2396 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:13.139681  2283 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.154086  2248 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:13.399750  2520 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.400190  2248 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:13.390210 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.401762 31065 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:13.401245  2248 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.182787  2547 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:13.243093 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.154732  2289 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:13.403100 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 897) took 1015 ms. Trace:
I20260709 12:25:13.403227 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:13.400786  2512 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:12.831338 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 106891us
I20260709 12:25:13.404310 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:13.296248 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 892) took 1230 ms. Trace:
W20260709 12:25:13.364439 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.405371 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:13.405709  2283 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:13.365576 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:13.367336 30958 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.012s	sys 0.009s
W20260709 12:25:13.406388 30958 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.012s	sys 0.009s
I20260709 12:25:13.406297  2283 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:13.406956 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 669951us
I20260709 12:25:13.342116 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.407032 31061 rpcz_store.cc:276] 0709 12:25:12.065884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.616890 (+551006us) service_pool.cc:224] Handling call
0709 12:25:12.639465 (+ 22575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.639806 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.639848 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.282044 (+642196us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b2c00000a78. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:25:13.282452 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.294495 (+ 12043us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:13.294536 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.294605 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.295980 (+  1375us) raft_consensus.cc:1685] finished
0709 12:25:13.295997 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.296198 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4426496,"thread_start_us":11687,"threads_started":1,"wal-append.queue_time_us":12133}
I20260709 12:25:13.408217  2490 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:13.408627 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.408933 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.409451 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.409708 30983 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:13.410254 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.410488  2278 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:13.411495  2130 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:13.411907  2130 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:13.369176  2543 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.640s	user 0.003s	sys 0.000s
I20260709 12:25:13.373018  2131 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:13.412707  2130 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.413321  2131 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:13.413898  2130 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.414163  2131 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.415402  2131 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.415524 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.416189 30986 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:13.408656  2490 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.417412  2490 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:13.377513  2545 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.544s	user 0.009s	sys 0.004s
I20260709 12:25:13.404095  2289 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:13.164268  2546 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:13.418673 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:13.406452 31689 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:13.419785  2336 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:13.420178  2336 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.420516  2336 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.421561 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:13.421990 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.422128 30987 raft_consensus.cc:2376] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9. Re-sending same reply.
I20260709 12:25:13.422595 31062 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:13.401867  2520 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.411392  2278 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:13.400305 31144 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:13.423516 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.424149 31062 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:13.424194 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.371486 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.424988 31142 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:13.425064 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.424506 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 1922) took 1699 ms. Trace:
I20260709 12:25:13.427495 31144 rpcz_store.cc:276] 0709 12:25:11.724839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.139855 (+415016us) service_pool.cc:224] Handling call
0709 12:25:13.424323 (+1284468us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003f48d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:13.424436 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866061056}
W20260709 12:25:13.361665 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.068s	sys 0.163s
W20260709 12:25:13.429687 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.069s	sys 0.163s
W20260709 12:25:13.429986 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 706889us
I20260709 12:25:13.430598 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:13.410823 31061 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.403651  2547 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.409569 31143 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:13.435297  2539 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:13.432839  2303 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:13.446007  2303 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.446435  2303 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.435817  2436 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.450627  2436 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.452787  2436 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:25:13.456609  2490 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.458796 31696 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:13.464174 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.465651 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.465986 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:13.466602 31137 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.466733 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:13.467376 31143 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.403460 31065 rpcz_store.cc:276] 0709 12:25:12.388139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.402379 (+1014240us) service_pool.cc:224] Handling call
0709 12:25:13.402638 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.402796 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.402822 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.402862 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:13.402895 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.402945 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.403049 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.477032 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.476951  2303 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:13.436995 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:13.477792 31065 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:13.286032 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2368) took 1130 ms. Trace:
I20260709 12:25:13.478394 31067 rpcz_store.cc:276] 0709 12:25:12.155877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.278652 (+1122775us) service_pool.cc:224] Handling call
0709 12:25:13.278992 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.279343 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.279377 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.279909 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.280997 (+  1088us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:13.281036 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.281106 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.285782 (+  4676us) raft_consensus.cc:1685] finished
0709 12:25:13.285808 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.285951 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"thread_start_us":558,"threads_started":1,"wal-append.queue_time_us":1067}
I20260709 12:25:13.435781 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:13.480103 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:13.480176 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44458.4us
I20260709 12:25:13.438740  2278 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:13.440436  2492 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:13.480641  2472 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.480846  2278 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:13.481024  2492 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:13.481818  2278 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.482432  2485 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.483078  2539 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:13.483848  2539 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:13.484215  2393 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.441691  2528 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:13.484923  1995 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:13.486109 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:13.442624 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:13.487747  2289 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.488142 31694 raft_consensus.cc:2368] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11.
I20260709 12:25:13.441155 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:13.487771  2283 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.489243 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:13.489459 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 52541.3us
I20260709 12:25:13.489718  2528 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:13.490165  2528 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:13.490406  2396 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:13.442881  2336 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:13.477458 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.490834  2396 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.491307  2396 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.491066  2528 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:13.493109  2336 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.484803 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.483016  2514 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:13.493836 31064 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:13.493063 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.492110  2528 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:13.439476 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 587828us
I20260709 12:25:13.495508 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.497087 31138 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:13.426173 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.493964  2514 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.498181  2514 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.498368 30987 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:13.483134  2492 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:13.429157 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.484803  2539 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.500741 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.501297 31062 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.483048  2278 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.502808  2502 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:13.504755 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:13.489907 31140 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:13.493163 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.506860 31136 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:13.509595 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:13.509902 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 84413us
I20260709 12:25:13.510802 30991 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:13.511158 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:13.480682 31067 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.512784 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:13.513285 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.514003 30990 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:13.515236  2283 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.515084 30991 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:25:13.518582 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:13.518891 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.519241 30983 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.515084  2289 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:13.519737 30987 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:13.405371  2512 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.529582  2283 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:13.532151 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2093) took 1303 ms. Trace:
I20260709 12:25:13.540005 31064 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:13.540208 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.534799  2502 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:13.555639 31137 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:13.560377 30983 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:13.712540 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:13.561623 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 66284.3us
I20260709 12:25:13.454669 31061 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:13.529412 31143 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:13.720292 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:13.582474 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.720871 31137 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.512013  2514 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:13.721159 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.721784 31143 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:13.722576 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.722846  2514 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:13.724063 31109 leader_election.cc:336] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.788s (SENT)
I20260709 12:25:13.719554  2502 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.724522 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:13.593976 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.725322 31138 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:13.725467 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.586593 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.726130 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:13.726720 31138 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.727013 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.727553 31065 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.594594 31067 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:13.595728 31062 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:13.597250 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:13.598050 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:13.731163 30992 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.731245 31136 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:13.602913 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:13.604362 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 167609us
I20260709 12:25:13.617882 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:13.732093 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 129280us
I20260709 12:25:13.615706 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:13.620975 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:13.625788 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:13.732949 31696 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.637823 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:13.641788 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:13.734149 31698 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:13.736660 31137 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:13.737566 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:13.737854 31138 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:13.738287 31137 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:13.738722 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.735332 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:13.739214 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.739764 31689 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:13.652967 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 896) took 1495 ms. Trace:
I20260709 12:25:13.535738 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:13.700040  2544 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.818436  2544 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.820696  2544 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.821200  2544 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:13.823271  2443 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.823751  2443 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.825891  2443 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.831296 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:13.831894 30985 raft_consensus.cc:2376] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9. Re-sending same reply.
I20260709 12:25:13.712540 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.715329 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2366) took 1596 ms. Trace:
I20260709 12:25:13.833381 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.832655 31058 rpcz_store.cc:276] 0709 12:25:12.118726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.866714 (+747988us) service_pool.cc:224] Handling call
0709 12:25:12.867024 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.399049 (+532025us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:13.421284 (+ 22235us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.605246 (+183962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.605323 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:13.605375 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.714231 (+108856us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b1800360ac0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87752a6 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419
0709 12:25:13.715106 (+   875us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54002cf1d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:13.715160 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.715270 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7018,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":646,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":1234810624,"thread_start_us":202855,"threads_started":2,"wal-append.queue_time_us":283619}
I20260709 12:25:13.834960 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2098) took 1114 ms. Trace:
I20260709 12:25:13.835544  2443 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.836305  2443 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.838266  2443 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:13.839119  2443 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:13.720373 31063 raft_consensus.cc:2368] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:25:13.839329  2303 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.710160 31059 rpcz_store.cc:276] 0709 12:25:12.228876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.605370 (+376494us) service_pool.cc:224] Handling call
0709 12:25:13.531984 (+926614us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400397bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:13.532090 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423127808}
I20260709 12:25:13.840422 31063 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:13.840981 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2099) took 1114 ms. Trace:
I20260709 12:25:13.724684  2502 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.841468 31059 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:13.842047 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 901) took 1050 ms. Trace:
I20260709 12:25:13.842368 31059 rpcz_store.cc:276] 0709 12:25:12.791307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.840920 (+1049613us) service_pool.cc:224] Handling call
0709 12:25:13.841184 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.841833 (+   649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.841995 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.740715 31068 rpcz_store.cc:276] 0709 12:25:12.157758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.393346 (+1235588us) service_pool.cc:224] Handling call
0709 12:25:13.393643 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.393833 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:13.618757 (+224924us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.652048 (+ 33291us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.652477 (+   429us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.652540 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:13.652601 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.652727 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.652890 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12175,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":218,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":147,"thread_start_us":256697,"threads_started":2}
I20260709 12:25:13.744220 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.746649 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:13.712085  2492 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.749758 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:13.753686 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:13.844347 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 91665.4us
I20260709 12:25:13.845047 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:13.754370  2578 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:13.850613 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 117931us
I20260709 12:25:13.851315 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:13.754793  2514 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:13.755292  2584 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:13.755750 31065 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:13.771637 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:13.749284 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:13.871917 31139 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:13.786266  2248 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.794770 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.874126 31067 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.786805  2566 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.803787  2563 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:13.789091  2416 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:13.813853 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 278190us
W20260709 12:25:13.701572 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.837805 31696 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:13.638432 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:13.848351  1995 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:13.849408 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:13.834968 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2097) took 1130 ms. Trace:
I20260709 12:25:13.835294 31058 rpcz_store.cc:276] 0709 12:25:12.721023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.834328 (+1113305us) service_pool.cc:224] Handling call
0709 12:25:13.834576 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.834709 (+   133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.834729 (+    20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.834761 (+    32us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:13.834788 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.834826 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.834907 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.841256 31063 rpcz_store.cc:276] 0709 12:25:12.726709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.839854 (+1113145us) service_pool.cc:224] Handling call
0709 12:25:13.840126 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.840746 (+   620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.840917 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:13.867303 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 128349us
I20260709 12:25:13.867965 31136 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:13.826210 30992 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
W20260709 12:25:13.880488 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 108967us
I20260709 12:25:13.879756  2248 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:13.881574 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.882828  2590 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:13.887817  2590 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:13.888064 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:13.888407  2248 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.889037  2248 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:13.888741  2590 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:13.889380  2248 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.889753  2248 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.886332 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:13.885413 31064 rpcz_store.cc:276] 0709 12:25:12.704172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.834272 (+1130100us) service_pool.cc:224] Handling call
0709 12:25:13.834537 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.834686 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.834709 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.834745 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:13.834773 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.834822 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.834907 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:13.897404 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48051.2us
I20260709 12:25:13.870368  2569 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:13.887203 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:13.889748 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:13.903075 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.907624 31109 leader_election.cc:400] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11.
I20260709 12:25:13.908135 31109 leader_election.cc:403] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:13.870690  2492 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.873593  2514 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:13.935421 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48292.4us
I20260709 12:25:13.936450 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:13.938241 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 88918.6us
I20260709 12:25:13.938869 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:13.874074  2581 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:25:13.942481 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:13.945995 31106 leader_election.cc:400] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:25:13.946347 31106 leader_election.cc:403] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:13.886772  2416 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.911015  2303 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:13.911103  2539 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:13.881577  2566 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:13.947583 31067 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:13.949631 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:13.953889 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:13.957743  2582 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:25:13.959918  2514 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:13.960510  2514 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11.
I20260709 12:25:13.961139  2514 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:13.947365  2588 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:13.963790  2588 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:13.967059  2566 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:13.968888  2566 raft_consensus.cc:2764] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:13.971685  2492 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:13.972272  2492 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8.
I20260709 12:25:13.944787  2553 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.945147  2569 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:13.990484 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:13.991014 31067 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:14.001077 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 128386us
I20260709 12:25:14.004194 31139 raft_consensus.cc:2410] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:25:13.954998 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.008028 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.016161 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:14.016976 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.018882 31059 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:25:14.022408 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 136180us
I20260709 12:25:14.023779  2596 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:14.023845 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:14.024611  2596 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:14.025197  2596 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:13.954208  2591 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:14.005801  2547 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:13.956000  2544 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:14.029698 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.014050  2539 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:14.047056 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:14.005928  2584 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:14.035018 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 80091.5us
I20260709 12:25:14.027724  2602 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:25:14.033350 31067 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:14.039832  2591 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.044231  2553 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:14.045418  2490 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:14.048130 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.021144  2303 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:14.068285 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73072.9us
I20260709 12:25:14.039218  2289 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:14.073055  2248 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:14.073808  2490 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:14.047916  2443 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:14.079804 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:14.069334  2396 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.000s
I20260709 12:25:14.083413 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:14.085649 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61870.8us
I20260709 12:25:14.087435 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:14.087927  2507 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:14.088112  2248 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:14.088802  2396 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:14.089293  2584 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:14.091486  2396 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:14.093182 31143 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:14.096148 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:14.096693 31060 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.080606  2490 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.083802  2283 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:14.101995 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 187782us
I20260709 12:25:14.102965 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.103466 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.099454  2602 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.108738  2591 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:14.039692  2596 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.085561  2443 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:14.116006 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.085296  2600 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.094745  2507 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:14.117434 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:14.123577  2336 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:14.117784 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.114619 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:14.100508  2547 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:14.127472  2278 consensus_meta.cc:287] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.001s
I20260709 12:25:14.127787  2547 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:14.128660  2547 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.130645  2443 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.119282 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.120229 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:14.101594  2490 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.122049 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:14.135739 31059 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:14.122654 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:14.102114  2283 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:14.134876 31060 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:14.113320  2591 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:14.136364 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.117122  2602 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:14.103154  2236 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:25:14.145417  2393 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.111382  2502 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.142055  2283 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.152122 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:14.137739  2512 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:14.138126  2416 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:14.154867  2507 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.147706  2595 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:14.147850  2406 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:25:14.118057  2303 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:14.163720  2472 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:14.145707  2278 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:25:14.145900  2578 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:14.183841  2578 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:14.184146  2602 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:14.184700  2602 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:14.184970  2578 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:14.185596  2602 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.137481  2590 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:14.187122  2590 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.187569  2590 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.168517 31061 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:14.148309  2553 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:14.191970  2553 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:14.192721  2592 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.192871  2553 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.193246  2592 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.195490  2592 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:14.196354  2289 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:25:14.197134 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45814us
I20260709 12:25:14.197793 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.198652  2416 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:14.199018  2416 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.199348  2416 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.200217 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.201025  2406 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:14.199013  2597 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:25:14.203289 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.203980  2472 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:14.204799  2472 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:14.206429  2336 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:14.208225  2528 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.002s
I20260709 12:25:14.180717 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.196405  2592 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:14.146447  2591 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.222566  2591 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.223959 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:14.224656 31143 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:14.226584 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:14.227890 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:14.228420 31141 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.228510 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.166009  2502 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:14.214875 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.206071  2472 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:14.215340  2595 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:25:14.221645 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.149554  2604 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:14.217293 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:14.229171 31691 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
W20260709 12:25:14.233796 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 53132.2us
I20260709 12:25:14.235481 31144 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:14.235868 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.236044  2241 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:14.236929 31694 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:25:14.237776 31692 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:14.246270 31690 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:25:14.237898 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.247349 30986 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:14.238546 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.248231 30983 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:14.239179 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.239785 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.242097 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:14.218184  2567 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.577s	user 0.000s	sys 0.004s
I20260709 12:25:14.235908 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.249560 31693 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:14.246141 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:14.253603 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:14.253486 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:14.259425 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:14.262943 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:14.235426 30878 maintenance_manager.cc:419] P 0423d91b75f1428faccc573fcbf419a1: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033376
W20260709 12:25:14.304193 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 50803.1us
I20260709 12:25:14.304571 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:14.309911 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.231732  2289 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:14.317298 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 63761us
I20260709 12:25:14.323228 31141 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:14.343144 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.342248 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.344470 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.341943 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.329526 31692 raft_consensus.cc:2410] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:14.255326  1889 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:14.356376 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:14.352193 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:14.352301 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:14.302999  2502 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:14.361114 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:14.309029  2236 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:14.365813 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.329175  2589 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:14.372840 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.313827  2406 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:14.374379 30985 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:14.314706  2604 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:14.381984 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.352797  2303 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.364818  2502 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:14.387722  2581 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:14.381150  2396 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.381479  2336 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:14.385432  2589 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:14.426445 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 379440us
I20260709 12:25:14.352782  2289 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.417037  2502 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:14.471834 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.475024 30983 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:14.417448  2581 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:14.475723 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 132640us
I20260709 12:25:14.493244 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.460421  2595 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:14.496855 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.507526 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 1932) took 1620 ms. Trace:
I20260709 12:25:14.498356 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:14.510900 31139 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:14.500080 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:14.460433  1889 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:14.457656  2602 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:25:14.516976 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.520323 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.521104 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.521593 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:14.531857 30885 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 314) took 18588 ms (client timeout 15000 ms). Trace:
I20260709 12:25:14.522413  2289 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:14.522538 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:14.523077  2485 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:14.477911  2581 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:14.535168 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.544054 31691 raft_consensus.cc:2410] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:14.534929 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:14.544942 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.545806 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:14.535554 31719 debug-util.cc:398] Leaking SignalData structure 0x7b08002f3fc0 after lost signal to thread 29848
I20260709 12:25:14.538702 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:14.547464 31719 debug-util.cc:398] Leaking SignalData structure 0x7b08002f3dc0 after lost signal to thread 30924
I20260709 12:25:14.548653  2520 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:14.549302  2609 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:14.549513  2520 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:14.539995 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:14.548709 31719 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4580 after lost signal to thread 31016
I20260709 12:25:14.551901 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:14.552074  2485 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:14.552285 31719 debug-util.cc:398] Leaking SignalData structure 0x7b0800270320 after lost signal to thread 31093
W20260709 12:25:14.552680 31719 debug-util.cc:398] Leaking SignalData structure 0x7b0800381320 after lost signal to thread 31094
I20260709 12:25:14.553165 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:14.553864  2592 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:14.554394 31692 raft_consensus.cc:2368] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13.
I20260709 12:25:14.554343  2592 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:14.554864  2592 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:14.555481  2607 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:25:14.540254  2547 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:25:14.548769 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.534931 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:14.557802 31140 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:14.540076 31697 raft_consensus.cc:2368] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
W20260709 12:25:14.553838 31719 debug-util.cc:398] Leaking SignalData structure 0x7b080032b280 after lost signal to thread 31168
I20260709 12:25:14.539083 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.559924 31698 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:14.539898 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.538623 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.495931  2506 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:14.455971  2396 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.570176 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:14.559912 31719 debug-util.cc:398] Leaking SignalData structure 0x7b080036dc80 after lost signal to thread 31722
I20260709 12:25:14.565064 31143 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:14.571241 31719 debug-util.cc:398] Leaking SignalData structure 0x7b0800298f40 after lost signal to thread 31723
W20260709 12:25:14.573855 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 191937us
I20260709 12:25:14.578346 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.581296  2623 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.588310 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:14.589198 31697 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 6: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:25:14.541110  2614 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:25:14.508812 31135 rpcz_store.cc:276] 0709 12:25:12.887551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.888380 (+   829us) service_pool.cc:224] Handling call
0709 12:25:13.293923 (+405543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.449533 (+155610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.449575 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.450163 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.562173 (+112010us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:13.596762 (+ 34589us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.639853 (+ 43091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.639932 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.201621 (+561689us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:14.493034 (+291413us) raft_consensus.cc:1685] finished
0709 12:25:14.507289 (+ 14255us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.507457 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155119,"prepare.queue_time_us":33172,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":214144,"raft.run_cpu_time_us":5954,"raft.run_wall_time_us":27759,"spinlock_wait_cycles":18505088,"thread_start_us":154332,"threads_started":2,"wal-append.queue_time_us":162220}
I20260709 12:25:14.617884 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:14.526252  2539 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:14.539216  2336 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:14.574673  2278 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:14.620775 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:14.622910 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44622us
I20260709 12:25:14.644459 31141 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:14.646595 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.649777 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:14.572773  2406 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:14.667825 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.615213 30887 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.676172 30987 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:25:14.685393 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.000s	sys 0.013s
I20260709 12:25:14.620051  2617 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:14.685702 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.000s	sys 0.013s
I20260709 12:25:14.626936  2569 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:14.695075 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 344699us
I20260709 12:25:14.695753 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.642436  2278 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:14.642460  2336 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.678308 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:14.656824  2406 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:14.686043 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 591368us
W20260709 12:25:14.692873  2490 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.000s
I20260709 12:25:14.707176 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:14.685045  2492 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:14.709141 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:14.708173 30887 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 6 to 7, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 7 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.693296  2617 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:14.701769  2528 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:14.719035 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:14.704077  2336 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:14.532405 30885 rpcz_store.cc:269] 0709 12:24:55.904396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.904615 (+   219us) service_pool.cc:224] Handling call
0709 12:24:57.337516 (+1432901us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fe090 00007f2eeb89cf3a 00007f2ef1652e9c 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:14.476317 (+17138801us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:11.737793 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:11.787969 (+ 50176us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:11.788025 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:25:11.788057 (+    32us) tablet.cc:662] Decoding operations
0709 12:25:11.789549 (+  1492us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:11.789628 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:11.789670 (+    42us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:25:12.119672 (+330002us) tablet.cc:701] Row locks acquired
0709 12:25:12.129556 (+  9884us) write_op.cc:260] PREPARE: finished
0709 12:25:12.129738 (+   182us) write_op.cc:270] Start()
0709 12:25:12.129911 (+   173us) write_op.cc:276] Timestamp: P: 1783599912129675 usec, L: 0
0709 12:25:12.129933 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:25:12.138581 (+  8648us) log.cc:844] Serialized 6279 byte log entry
0709 12:25:12.834464 (+695883us) op_driver.cc:464] REPLICATION: finished
0709 12:25:13.168733 (+334269us) write_op.cc:301] APPLY: starting
0709 12:25:13.169079 (+   346us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:13.172153 (+  3074us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:13.172299 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:13.174609 (+  2310us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:13.175010 (+   401us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 12:25:13.175076 (+    66us) write_op.cc:312] APPLY: finished
0709 12:25:13.175643 (+   567us) log.cc:844] Serialized 113 byte log entry
0709 12:25:13.175831 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:13.175996 (+   165us) write_op.cc:454] Released schema lock
0709 12:25:13.176073 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:13.176493 (+   420us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:13.177777 (+  1284us) write_op.cc:454] Released schema lock
0709 12:25:13.393224 (+215447us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb89132a 00007f2eeb816f14 00007f2eeb816ec3 00007f2eeb816e38 00007f2eeb816df3 00007f2eeb8160dc 00007f2eeb83fe16 00007f2eeb827cbb 00007f2eeb81239d 00007f2eeb815d75
Metrics: {"mutex_wait_us":1440882,"reported_tablets":45,"spinlock_wait_cycles":14910208,"child_traces":[["op",{"apply.queue_time_us":332675,"apply.run_cpu_time_us":4633,"apply.run_wall_time_us":7972,"num_ops":16,"prepare.queue_time_us":505926,"prepare.run_cpu_time_us":22872,"prepare.run_wall_time_us":771144,"raft.queue_time_us":181314,"raft.run_cpu_time_us":57795,"raft.run_wall_time_us":594569,"replication_time_us":704335,"spinlock_wait_cycles":8784256,"thread_start_us":1151348,"threads_started":4,"wal-append.queue_time_us":392941,"wal-append.run_cpu_time_us":16377,"wal-append.run_wall_time_us":1643457}]]}
W20260709 12:25:14.691973 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 111523us
I20260709 12:25:14.677438 31143 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:14.665714  2131 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:14.695835 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:14.568881  2547 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:25:14.729666  2131 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.730080  2131 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.688447 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.702596  2609 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.731914  2588 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:14.732975 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44636us
W20260709 12:25:14.671725  2584 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.006s	sys 0.003s
I20260709 12:25:14.707123  2406 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:14.734395  2584 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.006s	sys 0.004s
I20260709 12:25:14.705127  1995 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:14.726281 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:14.733917  2588 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:25:14.742949 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.742936 30887 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 5, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 5 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.743129 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:14.696816  2436 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:25:14.540752  2604 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.759029 30887 catalog_manager.cc:5697] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 2 to 6, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.693560  1889 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:14.715777  2490 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.678160  2589 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:14.762226 31136 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.025s	sys 0.020s
W20260709 12:25:14.845284 31136 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.042s	sys 0.053s
I20260709 12:25:14.708779  2485 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:14.852223  2393 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.000s
I20260709 12:25:14.820837 31693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48047b2b139146c38a821328cf7bd148"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
W20260709 12:25:14.820344 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58987.2us
I20260709 12:25:14.719683  2130 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:14.864270 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 168490us
I20260709 12:25:14.921797 31693 raft_consensus.cc:606] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:14.924211 31109 leader_election.cc:336] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.687s (SENT)
W20260709 12:25:14.822088  2571 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:25:14.937281 31693 raft_consensus.cc:614] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:14.949970 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:14.941262 31140 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:25:14.784063  2611 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.363s	user 0.000s	sys 0.002s
I20260709 12:25:14.768843  2547 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:14.961074 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.923019  2393 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.787308  2584 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.176s
I20260709 12:25:14.978333 31661 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:14.960578  2618 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:14.787886  1995 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:14.942610 30887 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:14.967576  2612 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.345s	user 0.010s	sys 0.000s
W20260709 12:25:14.991292 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 67109.4us
W20260709 12:25:14.992305 31062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.003s	sys 0.003s
W20260709 12:25:14.999696 31062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.003s	sys 0.004s
W20260709 12:25:14.954654 31696 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.008s	sys 0.015s
I20260709 12:25:14.963750  2547 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:15.001276 31696 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.008s	sys 0.015s
I20260709 12:25:14.949882  2130 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.971365  2623 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:14.989108  2283 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.822s	user 0.011s	sys 0.000s
I20260709 12:25:14.847716  2406 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.956449  2131 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:14.980353  2625 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:15.008708 30887 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.988768  2396 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:15.049659 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 64990.6us
I20260709 12:25:14.992754  2618 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:15.036583  2613 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260709 12:25:15.026955  2547 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:15.063534 30887 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.090247 31068 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:15.090530 30987 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:15.091399 31059 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:25:15.057376  2619 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.348s	user 0.000s	sys 0.001s
I20260709 12:25:15.051143  2520 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:14.936923  2617 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.084933  2618 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:15.120074 30990 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
W20260709 12:25:15.122185 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 123072us
I20260709 12:25:15.051548  2283 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.125669 30887 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 5 to 7, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 7 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:15.127475  2645 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.110910  2472 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:15.135407 30986 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:15.136035 30887 catalog_manager.cc:5697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 5 to 7, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 7 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:15.143625  2638 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 12:25:15.151418  2416 consensus_meta.cc:287] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.952s	user 0.005s	sys 0.006s
I20260709 12:25:15.151878  2416 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.154054 30992 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:15.154809  2648 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:15.155079  2416 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:15.155854  2633 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
I20260709 12:25:15.140245  2472 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:15.157912  2615 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.684s	user 0.002s	sys 0.000s
I20260709 12:25:15.138618  2645 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:15.168975  2639 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 12:25:15.171132  2590 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.983s	user 0.014s	sys 0.000s
I20260709 12:25:15.171691 30887 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 7 to 8, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.173540  2131 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:25:15.183270 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:15.183359  2406 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:15.247437 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 65800.7us
W20260709 12:25:15.194119  2592 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.639s	user 0.005s	sys 0.005s
W20260709 12:25:15.189931  2553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.002s	sys 0.007s
I20260709 12:25:15.252876 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:15.195066  2283 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.195027  2617 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:15.256884 31136 raft_consensus.cc:606] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:15.282483 31136 raft_consensus.cc:614] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:15.282727 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 231929us
W20260709 12:25:15.212188  2591 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.973s	user 0.008s	sys 0.003s
I20260709 12:25:15.195433  2130 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.191583  2539 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:15.272194 31065 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:25:15.256665  2553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.003s	sys 0.007s
I20260709 12:25:15.273377 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:15.262826  2616 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.802s	user 0.003s	sys 0.000s
I20260709 12:25:15.257619 30887 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 8 to 9, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 9 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:15.255443  2640 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.147s	user 0.000s	sys 0.003s
I20260709 12:25:15.250593  2569 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.286268 31061 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:15.294377 31061 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:15.263919  2581 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.721s	user 0.005s	sys 0.005s
I20260709 12:25:15.286942 31058 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:15.288650 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:15.194790  2336 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.311925 31061 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.288504  2539 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:15.257359  2592 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.256385  2590 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:15.328461 31695 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.007s	sys 0.017s
I20260709 12:25:15.335341 30983 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:15.337564 31695 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.008s	sys 0.017s
I20260709 12:25:15.335752 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 918) took 1108 ms. Trace:
I20260709 12:25:15.335569 30887 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 6, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.338515 31061 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:15.340567 31696 rpcz_store.cc:276] 0709 12:25:14.227197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.232244 (+  5047us) service_pool.cc:224] Handling call
0709 12:25:14.232577 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.232815 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:14.242671 (+  9856us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.334818 (+1092147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:15.334990 (+   172us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:15.335049 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.335588 (+   539us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.335698 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5198,"prepare.queue_time_us":10017,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":82526,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":3502,"thread_start_us":1091510,"threads_started":2,"wal-append.queue_time_us":890532}
I20260709 12:25:15.344348 31138 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:15.350291 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:15.287904  2591 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.350921 31138 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:15.352851 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:15.353444 30985 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:15.358548 31136 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:15.301368  2393 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:25:15.361492 31140 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:15.362658 30991 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:15.304917  2604 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:25:15.358731 30887 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 7 to 9, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 9 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:15.366053 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 52025.9us
I20260709 12:25:15.314613  2581 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.363530 31136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:15.376402 31136 raft_consensus.cc:606] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:15.376748 31136 raft_consensus.cc:614] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:15.349318  2599 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:25:15.373531  2336 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:15.373701  2592 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:15.375042  2645 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.384631  2472 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:15.398468 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:15.314850  2569 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.399629 30887 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 5 to 6, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:15.477120 31139 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:25:15.433073  2651 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:25:15.426191  2520 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:15.436946 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 60017.8us
I20260709 12:25:15.485582 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:15.437954  2654 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:15.439185  2528 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:15.383208  2588 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.442528 31061 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:15.443042 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2125) took 1094 ms. Trace:
I20260709 12:25:15.489984 31062 rpcz_store.cc:276] 0709 12:25:14.348231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.348345 (+   114us) service_pool.cc:224] Handling call
0709 12:25:14.348654 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.000708 (+652054us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:15.086262 (+ 85554us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.291492 (+205230us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.291702 (+   210us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.305440 (+ 13738us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.305592 (+   152us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:15.305621 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.305681 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.442838 (+137157us) raft_consensus.cc:1685] finished
0709 12:25:15.442858 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.442985 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40024,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":4021,"raft.queue_time_us":45036,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":30488,"spinlock_wait_cycles":13824256,"thread_start_us":824538,"threads_started":2,"wal-append.queue_time_us":255001}
I20260709 12:25:15.443756 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 917) took 1217 ms. Trace:
I20260709 12:25:15.446597 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:15.448186  2581 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:15.448211 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:15.475739  2502 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.978s	user 0.010s	sys 0.002s
I20260709 12:25:15.451313 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2338) took 1083 ms. Trace:
I20260709 12:25:15.401994 30986 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.369061  2649 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.456130  2278 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:15.462339  2631 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.641s	user 0.007s	sys 0.004s
I20260709 12:25:15.462484 31144 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:15.467047  2590 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.482705 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:15.486536 31143 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:15.495075 31136 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.448980 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:15.498646 30887 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 9, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 9 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.500195  2588 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.488795  2528 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:15.501973 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:15.503461  2654 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:15.502607  2644 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:15.501956 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:15.506489  2658 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:15.516590 31137 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:15.499785  2623 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:15.536123 30887 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 5 to 6, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 6 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:15.512707 30984 rpcz_store.cc:276] 0709 12:25:14.367673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.367815 (+   142us) service_pool.cc:224] Handling call
0709 12:25:14.380886 (+ 13071us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.382628 (+  1742us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:14.631469 (+248841us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.781302 (+149833us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.781859 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.782039 (+   180us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:14.782296 (+   257us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.782500 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.782588 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.171378 (+388790us) spinlock_profiling.cc:243] Waited 9.93 ms on lock 0x7b18002e80a0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:15.430813 (+259435us) raft_consensus.cc:1685] finished
0709 12:25:15.430851 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.451255 (+ 20404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20237,"prepare.queue_time_us":73598,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":88,"raft.queue_time_us":27326,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":3313,"spinlock_wait_cycles":71899648,"thread_start_us":283373,"threads_started":2,"wal-append.queue_time_us":309261}
I20260709 12:25:15.521731 31138 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.493185 31695 rpcz_store.cc:276] 0709 12:25:14.226861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.233066 (+  6205us) service_pool.cc:224] Handling call
0709 12:25:14.240418 (+  7352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.240637 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:14.718393 (+477756us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.340421 (+622028us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.401146 (+ 60725us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.443457 (+ 42311us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:15.443502 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:15.443549 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.443610 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.443704 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89087,"prepare.queue_time_us":108221,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":68,"raft.queue_time_us":131610,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":9477,"spinlock_wait_cycles":12904832,"thread_start_us":1024094,"threads_started":2,"wal-append.queue_time_us":655414}
I20260709 12:25:15.551872 31136 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:15.555531 30986 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:15.571043 30985 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:25:15.568032 30887 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:15.575392 30887 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 5 to 8, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 8 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:15.576689 30887 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:15.582888 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:15.583885 31696 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:15.590001 31697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60b6187f32774daba89763099dabeb2a"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:15.567945  2658 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.590652 31697 raft_consensus.cc:606] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:15.591228 31697 raft_consensus.cc:614] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:15.510746  2604 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:25:15.612255 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:15.592845 31138 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:15.598645  2620 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:25:15.599287  2657 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:15.599591  2528 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.580363  2278 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:15.601852 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.602438  2618 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.572013  2599 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:25:15.593786  2658 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:15.604473  2547 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 12:25:15.605095  2588 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:15.643056 31033 connection.cc:256] RPC call timeout handler was delayed by -1.6037484849998691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
W20260709 12:25:15.655146 31033 connection.cc:256] RPC call timeout handler was delayed by -1.5524484849997862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
W20260709 12:25:15.655462 31033 connection.cc:256] RPC call timeout handler was delayed by -1.4579484850000881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 12:25:15.655769 31033 connection.cc:256] RPC call timeout handler was delayed by -1.3796484849999615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 12:25:15.656037 31033 connection.cc:256] RPC call timeout handler was delayed by -1.2464484850002009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
W20260709 12:25:15.605435 30990 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2331) took 1294 ms (client timeout 1513 ms). Trace:
W20260709 12:25:15.656297 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0736484849999215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
W20260709 12:25:15.656406 30990 rpcz_store.cc:269] 0709 12:25:14.311317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.311481 (+   164us) service_pool.cc:224] Handling call
0709 12:25:15.605273 (+1293792us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540029a9d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:15.605373 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463552768}
I20260709 12:25:15.657646 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:15.658762 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:15.659843 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:15.660460 30990 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:15.661312 30990 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:15.641877  2539 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.664806  2539 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:15.631309  2657 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.665536  2657 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.645349  2553 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.605573  2664 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:25:15.606554 30988 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:25:15.606688 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2322) took 1392 ms. Trace:
I20260709 12:25:15.683156 30990 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:25:15.607028  2662 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:25:15.610569 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:15.683262 30992 rpcz_store.cc:276] 0709 12:25:14.214009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.214137 (+   128us) service_pool.cc:224] Handling call
0709 12:25:15.606444 (+1392307us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b5400225e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:15.606624 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2120941312}
I20260709 12:25:15.682610  2547 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.684306  2553 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.611523  2544 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:15.611867 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:15.612363  2569 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:15.613000 30985 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:25:15.520996  2502 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.614210 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:15.634366 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:15.542510  2649 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.638756  2130 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:25:15.639572  2591 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.640676  2490 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.632804 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:15.723634 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:15.707258 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50387.5us
I20260709 12:25:15.722288 30989 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:15.724846 31693 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:15.692087 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:15.725217 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:15.722730 30987 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:15.722340 31697 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:15.729045 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:15.733287 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:15.733383 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:15.653744  2645 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:25:15.736904  2502 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.738313 31064 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:15.739008 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:15.741577 31065 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:15.702466 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:15.742620 31062 raft_consensus.cc:2368] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10.
I20260709 12:25:15.744127 31068 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:15.745746 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:15.746927 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:15.747316 31692 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:15.747476 31698 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:15.748771 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:15.750232 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:15.746940  2502 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:15.757922  2670 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:25:15.727552  2528 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:15.688907  2553 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:15.722612 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 146710us
I20260709 12:25:15.769155 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:15.688706  2547 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.764598 31065 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:15.723394  2236 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:15.723584  2436 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:15.773628 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:15.766297 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:15.738700  2241 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:15.706807  2618 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.773746 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:15.775233 31068 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.786590  2539 raft_consensus.cc:2764] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 12:25:15.786633 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:15.788072 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:15.788998 31689 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:15.789201 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:15.787714 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:15.748301  2620 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.732609  2544 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.760504  2630 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:15.806710 31062 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:15.791702  2236 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.813228 31695 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:15.803476  2506 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:25:15.817813  2654 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:15.802655 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:15.791760  2588 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:15.851161 31063 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:15.833557  2630 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:15.892966 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 144248us
I20260709 12:25:15.888033  2236 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.852555  2654 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.898746 31698 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:15.900306 31068 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:15.832890  2620 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.903283 30987 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:15.901150  2630 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:15.906957  2673 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:25:15.908620 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2127) took 1415 ms. Trace:
I20260709 12:25:15.908984 31064 rpcz_store.cc:276] 0709 12:25:14.493324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.493501 (+   177us) service_pool.cc:224] Handling call
0709 12:25:14.493882 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.738809 (+1244927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.908409 (+169600us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540027c254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:15.908567 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2861503872}
W20260709 12:25:15.910941  2666 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 12:25:15.911283 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:15.912521  2589 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:15.912807 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 660) took 1335 ms. Trace:
I20260709 12:25:15.913137 30983 rpcz_store.cc:276] 0709 12:25:14.577442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.577915 (+   473us) service_pool.cc:224] Handling call
0709 12:25:15.912624 (+1334709us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400226854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:15.912755 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574539136}
W20260709 12:25:15.915109  2669 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
W20260709 12:25:15.917675  2665 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
I20260709 12:25:15.920394 30990 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:15.922361 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:15.929095 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 626) took 1429 ms. Trace:
W20260709 12:25:15.930647 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 676542us
I20260709 12:25:15.931504 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:15.918864  2236 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:25:15.929466 31142 rpcz_store.cc:276] 0709 12:25:14.499620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.512639 (+ 13019us) service_pool.cc:224] Handling call
0709 12:25:14.512983 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.513213 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:14.570540 (+ 57327us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.024246 (+453706us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b18003fda80. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:25:15.123361 (+ 99115us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.928651 (+805290us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b18003fda80. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:25:15.928788 (+   137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:15.928836 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:15.928883 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.928946 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.929041 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":248992,"prepare.queue_time_us":50314,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2447,"raft.queue_time_us":808200,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":4197,"spinlock_wait_cycles":24441216,"thread_start_us":97627,"threads_started":1,"wal-append.queue_time_us":684771}
I20260709 12:25:15.940200 31698 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:15.941515 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:15.942201 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:15.943840 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:15.944546 31137 raft_consensus.cc:2368] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:15.941443 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:15.792068  2436 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.948678 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:15.949577 31137 raft_consensus.cc:2368] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:15.951579 31137 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:15.952278 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:15.954816 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:15.956884 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:15.933313  2289 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:15.931778  2588 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:15.938428  2485 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:15.936610  2604 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.962313 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:15.947685  2581 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.967433 31137 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:15.950783  2625 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:15.950639  2512 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:15.972497 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.973675 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:15.974285 31064 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:15.975378  2512 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:15.975782 31139 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:25:15.977497  2512 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:15.978941 31137 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:15.975826 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:15.971755  2604 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.991984  2649 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:15.972872  2578 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:15.990780  2627 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:25:16.002810 31135 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:25:15.978480  2485 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:15.947804  2658 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:15.980125  2507 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:15.999872  2241 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:15.999815  2609 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:16.063427  2609 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.063877  2609 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:16.063947  2674 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:25:15.973346  2581 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:16.002115  2672 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:16.080667 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 132354us
I20260709 12:25:16.135463 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:16.138166 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:16.137336 31695 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:16.150396 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 150807us
I20260709 12:25:16.144093 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:16.157310 31063 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:25:16.177716 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.153574 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:16.205572 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.206832 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.207634 31138 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.209162 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.213759 31068 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:16.215360 31067 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:16.079483  2544 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:16.133955  2679 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:16.231571 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.232180 30991 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:16.234589 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:16.227667 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 73951.2us
I20260709 12:25:16.138060  2627 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:16.240762 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 297474us
I20260709 12:25:16.242895 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:16.252777 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.259881 31137 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:16.267529 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.268350 31137 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:16.269949 30983 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:16.212718  2512 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:16.215706  2625 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.067636  2507 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.281139 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.134756  2657 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.290321 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:16.281981 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.205185  2539 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:16.248004  2289 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.215027  2553 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 12:25:16.269860  2638 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:25:16.079020  2588 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.247206  2544 raft_consensus.cc:2764] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:16.298203 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.284932  2520 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:16.284725  2604 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.302466 31138 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:16.303421 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:16.303884 31140 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:25:16.325109 31140 pending_rounds.cc:85] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 7
I20260709 12:25:16.324949 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.325350 31140 pending_rounds.cc:107] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:25:16.230634  2618 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:25:16.249948  2679 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.315217  2620 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.318001 31692 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.150086  2672 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:16.298650  2659 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:25:16.298406  2685 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:16.325496 31138 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:16.325596 31140 raft_consensus.cc:2889] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.323135 31144 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:16.326398 31141 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:25:16.325551 31142 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:16.270753  2647 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:25:16.255141  2627 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:16.290760  2512 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.335348 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.280627  2485 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.313108  2289 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:16.409322 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.376556 31064 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:16.412307 31137 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.295938  2625 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.414588 31138 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
W20260709 12:25:16.376434 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 134375us
I20260709 12:25:16.410061 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:16.410266 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 153369us
I20260709 12:25:16.417199 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:16.330770  2678 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 12:25:16.437040 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:16.386041  2520 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.403126  2654 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.421015  2512 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:16.442587 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:16.430105  2658 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.430335  2604 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:16.421244  2289 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.460888 31137 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:16.444279  2645 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:16.447230  2443 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:16.448956  2520 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.484551  2679 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:16.485334  2679 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:16.453860  2581 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:16.463758  2604 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.485975  2679 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:16.493696  2604 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.495534  2679 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.496472  2627 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:16.487999  2443 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.519280  2443 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:16.520246  2688 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:25:16.522496  2696 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.526208  2625 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:16.529902  2657 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:16.553918  2682 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:25:16.560376 31692 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:16.523103  2696 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.608206  2507 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.622146 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 979) took 1053 ms. Trace:
I20260709 12:25:16.622766 31691 rpcz_store.cc:276] 0709 12:25:15.568526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.568642 (+   116us) service_pool.cc:224] Handling call
0709 12:25:15.572626 (+  3984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:15.585455 (+ 12829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:15.585481 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:15.588413 (+  2932us) log.cc:844] Serialized 26 byte log entry
0709 12:25:15.666046 (+ 77633us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b18004d8900. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:15.947517 (+281471us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:16.154832 (+207315us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.216369 (+ 61537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.216444 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.620286 (+403842us) raft_consensus.cc:1685] finished
0709 12:25:16.621915 (+  1629us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.622079 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94388,"prepare.queue_time_us":118705,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":75432,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":29977,"spinlock_wait_cycles":5056256,"thread_start_us":470121,"threads_started":2,"wal-append.queue_time_us":422525}
W20260709 12:25:16.667768 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 424900us
W20260709 12:25:16.668326 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.674753  2604 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:16.747607 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.758801 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:16.748649  2693 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 12:25:16.765416 31696 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:16.769065 31109 leader_election.cc:400] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13.
I20260709 12:25:16.769558 31109 leader_election.cc:403] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:16.747294  2696 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:16.765650  2700 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:25:16.779165  2278 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.763473  2597 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:16.791579  2278 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.801172  2278 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:16.779349  2708 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:16.806490 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:16.802371  2578 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.791s	user 0.012s	sys 0.000s
I20260709 12:25:16.820318  2520 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:16.801831  2604 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:16.828961 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:16.810820  2578 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:16.834338  2685 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:16.824676  2539 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:16.825220  2597 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:16.835640 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:16.835415  2528 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.057s	user 0.013s	sys 0.000s
I20260709 12:25:16.841552 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:16.845045 31109 leader_election.cc:400] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 5 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6.
I20260709 12:25:16.840382  2597 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.837582  2539 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:16.843696  2514 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:16.851437 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:16.851444 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:16.853665  2609 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.789s	user 0.010s	sys 0.000s
I20260709 12:25:16.846122  2685 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:16.855919 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:16.856338  2443 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.857877 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:16.859959 31691 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:16.857776 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:16.861776 31109 connection.cc:256] RPC call timeout handler was delayed by -1.2746515869998802s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 12:25:16.862128 31109 connection.cc:256] RPC call timeout handler was delayed by -1.1630515869996998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260709 12:25:16.868042 31698 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:16.866767  2578 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:25:16.866832  2514 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:16.869277  2514 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.871551  2514 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:16.849810  2597 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:16.880082 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:16.881450  2627 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:16.881661  2685 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:16.882522  2539 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:16.879873  2630 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.974s	user 0.013s	sys 0.000s
I20260709 12:25:16.883773  2539 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.884045  2630 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.881807  2436 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.887174 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.887727 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.888224 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:16.888293 31136 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:16.888809 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.889386 31138 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:16.890177 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.892632 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:16.893520 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.894098 31137 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:16.894098 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:16.854100  2609 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.896200 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:16.896919  2602 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:16.897056 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.897338  2706 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:16.897701 31143 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.897517  2602 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.898389  2710 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:16.914584 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.897848  2706 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:16.886088  2630 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:16.899756 31033 leader_election.cc:400] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:16.916817 31033 leader_election.cc:403] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:16.903116  2443 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.904186  2708 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13.
I20260709 12:25:16.904495  2703 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:16.909929 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.910199 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:16.910483 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.910723  2485 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:16.911026  2609 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.898871  2241 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.912774  2686 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:16.955864 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:16.958143 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.913627 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.917985 31033 leader_election.cc:400] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:16.880436  2514 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:25:16.923959 31136 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:16.943387 30984 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.944531 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.963236 30989 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:16.912163 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:16.946005 30987 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:16.947329 30986 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:16.900830  2436 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:16.925649  2710 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:16.948514 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.966037 31140 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:16.966497 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.967569 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.968575 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:16.968966 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 582) took 1080 ms. Trace:
I20260709 12:25:16.969532 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.944393  2703 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.971783  2706 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:16.972958  2703 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:16.969297 31695 rpcz_store.cc:276] 0709 12:25:15.888623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.888849 (+   226us) service_pool.cc:224] Handling call
0709 12:25:15.931510 (+ 42661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.144756 (+213246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.968752 (+823996us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54002f99d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:16.968912 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344459008}
I20260709 12:25:16.964746 31698 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:16.950781  2604 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:16.945206 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:16.955468 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:16.975314 31137 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.975543 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:16.919162 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:16.975791 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:16.963847 31143 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:16.926549  2647 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:16.976313 30988 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.901823  2578 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:16.964684  2581 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:16.954792 30983 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.959525 31033 leader_election.cc:403] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:16.978039 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:16.986140 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:16.986904 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:16.987401 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 69698.9us
I20260709 12:25:16.987766 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:16.988196 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:16.986708  2278 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:16.990096  2703 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:16.990931 31060 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:16.991086  2649 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:16.991900 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:16.992436  2539 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:16.992535 31060 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:16.993255  2539 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:16.993618 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:16.990025 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:16.995072 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:16.995515 31060 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:16.995606 31059 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:16.996380 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:16.996912 31060 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:16.997139 31140 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:16.997568 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:16.997691 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:16.998396  2706 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:16.983485  2520 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:16.983783  1995 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:25:17.000741  2679 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:17.001034  1995 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:17.001121  2679 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:16.980275  2686 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:16.976311  2436 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:16.956453  2485 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.004289 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:17.005026  2539 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:17.005277  2485 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.006633 31060 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:17.007709 31063 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:17.007680 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:17.008802 31060 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:25:16.987008  2528 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:25:16.987701 30985 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:16.991668  2649 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:16.998620 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:16.992743  2697 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.474s	user 0.000s	sys 0.001s
I20260709 12:25:17.013114 31137 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:16.964732  2609 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:17.011435 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:17.020886 31068 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:17.021956 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:16.976197  2514 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.032869 31059 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:17.002954  2436 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:16.986874  2657 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.036928 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:16.963778  2241 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.038560 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:17.044816 30983 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:17.020794  2609 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.012106  2710 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.012136  2708 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.042711 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:17.054934 30984 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
W20260709 12:25:17.059038 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 81111.1us
I20260709 12:25:16.990173  2278 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.067158 30983 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:25:17.067735 30983 raft_consensus.cc:606] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:17.036469  2602 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:17.069537 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:17.034684  2647 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:17.068114 30983 raft_consensus.cc:614] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:17.082301 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:17.083580 30984 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:17.083767 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:17.036418  2514 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.088595 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:17.002063  2679 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.085529 31695 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:17.078841  2654 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:17.060230  2625 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:17.047988  2709 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:17.047538  2696 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:25:17.056922  2609 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:17.069008  2544 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:17.061828  2708 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.092846  2703 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:17.048267  2657 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.096284 30983 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:17.085218  2630 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:17.092685  2654 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.105247 30988 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:17.081847  2710 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:25:17.088331  2602 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:17.093101  2709 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:17.088476  2698 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:17.094200  2625 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.102321  2520 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:17.114847 30984 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:17.103195  2544 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:17.134464 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 96192.4us
I20260709 12:25:17.108839  2679 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.108701  2702 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:17.140744 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:17.135458 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 46950us
I20260709 12:25:17.144140 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:17.115744  2654 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.116091  2588 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:17.144155  2721 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.147526  2721 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.191463 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2104) took 1506 ms. Trace:
I20260709 12:25:17.191968 30990 rpcz_store.cc:276] 0709 12:25:15.684950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.685778 (+   828us) service_pool.cc:224] Handling call
0709 12:25:15.918201 (+232423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.921108 (+  2907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.191228 (+1270120us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b5400226850. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:17.191399 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533902080}
W20260709 12:25:17.204427 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.150142  2520 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.134032  2703 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:17.210652  2703 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.144081 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:17.217052  2728 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:17.219375 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 75983.1us
I20260709 12:25:17.152182  2588 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:17.140297  2625 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:17.133832  2709 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:17.230007  2700 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:25:17.236482 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:17.239253 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:17.240703 30984 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:17.149776  2698 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:17.241825 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 105460us
I20260709 12:25:17.243636 31692 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:17.243726 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.246991 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:17.248843 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 103030us
I20260709 12:25:17.249907 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:17.251883 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.225178  2588 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.237833  2647 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:17.256922 31106 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:17.258603 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:17.252698 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:17.227435  2528 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:25:17.259694 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 936) took 1497 ms. Trace:
I20260709 12:25:17.262615 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.245018  2649 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.232746  2597 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.250353  2696 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:17.260838 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:17.264777 31690 rpcz_store.cc:276] 0709 12:25:15.762322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.788798 (+ 26476us) service_pool.cc:224] Handling call
0709 12:25:15.928378 (+139580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.963749 (+ 35371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:15.963795 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:15.963850 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:15.963893 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.259472 (+1295579us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b54005811d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:17.259516 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.259636 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80751488}
I20260709 12:25:17.266166 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.268443 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:17.268651 31695 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:17.269523 31697 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:17.270781 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:17.251260  2726 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:17.267659  2588 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.236480  2702 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:25:17.259816  2710 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.244355  2728 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:25:17.286952  2131 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:17.287496  2131 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:17.288341  2131 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.290059  2672 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.267246  2278 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.294906 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 579) took 1555 ms. Trace:
I20260709 12:25:17.295287 31696 rpcz_store.cc:276] 0709 12:25:15.739847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.760778 (+ 20931us) service_pool.cc:224] Handling call
0709 12:25:15.793002 (+ 32224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.766626 (+973624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.294693 (+528067us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54004d7e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:17.294851 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2235857152}
I20260709 12:25:17.286289 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:17.271893  2696 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.272058  2649 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:17.275107  2241 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.279799  2726 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:17.317584 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.318993 31135 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:17.285774  2490 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.318883  2649 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.298565  2737 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:25:17.259087  2544 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.335830  2578 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.336352  2544 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:17.339244  2722 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 12:25:17.342783  2724 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:17.297232  2721 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:17.343520  2647 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:17.347299  2734 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:17.304548  2732 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.348578  2732 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.350708  2732 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:17.310530  2520 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.351784  2703 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:17.319621  2729 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:25:17.353108  2729 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:17.353413 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2152) took 1074 ms. Trace:
I20260709 12:25:17.353941 31141 rpcz_store.cc:276] 0709 12:25:16.279062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.279823 (+   761us) service_pool.cc:224] Handling call
0709 12:25:16.326079 (+ 46256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.408926 (+ 82847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.353142 (+944216us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b5400094cd0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:17.353339 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105623808}
I20260709 12:25:17.355826 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.357133 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.299065  2528 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:17.300652  2696 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.358556 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 934) took 1223 ms. Trace:
I20260709 12:25:17.359036 31064 rpcz_store.cc:276] 0709 12:25:16.135013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.135182 (+   169us) service_pool.cc:224] Handling call
0709 12:25:17.358354 (+1223172us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54000fc654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:17.358486 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552841984}
I20260709 12:25:17.310698  2698 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.320442  2726 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.363265  2584 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:17.363642  2584 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:17.364424  2584 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.365604  2584 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:17.368745 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.372112  2602 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.358559  2597 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:17.332000 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45814.4us
I20260709 12:25:17.357235  2630 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:17.366008  2715 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.543s	user 0.002s	sys 0.000s
W20260709 12:25:17.378993 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 122170us
I20260709 12:25:17.380055  2743 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:17.379914 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:17.380473  2743 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:17.381233  2597 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:17.381716  2739 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:17.381752  2597 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.285491  2710 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.374899  2706 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.383430 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.385730 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:17.386565 31064 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:17.389806 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:17.390632 31064 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:17.390905 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.391469 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:17.330987  2657 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.332962  2708 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:17.398917 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:17.420485 31064 raft_consensus.cc:2410] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:17.402681  2528 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.375458  2490 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.410315  2706 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.411679  2702 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:17.414278 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:17.415025 31141 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:17.378674  2630 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.382481  2597 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.440407 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:17.382200  2739 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:17.425587 31135 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:25:17.353888  2609 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.436865 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:17.336387  2578 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:17.479197 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.474351 31697 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:17.487056 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:17.488211 31695 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:17.492614  2739 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.493837  2739 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.470660  2702 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.499480  2490 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.433647  2528 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.500954  2702 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.501528  2702 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:17.501748  2528 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:17.503396  2578 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:17.471407  2741 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.504210  2741 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.509089  2602 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:17.512778 31064 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:17.426326  2436 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.493927  2609 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.520818 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.381366  2743 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.523535 31141 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:17.524353 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:17.525468 31135 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:17.485045  2744 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.490681  2728 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:17.511169  2741 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:17.528719 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 270176us
I20260709 12:25:17.550702 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:17.554728  2737 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:17.555109  2737 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.555454  2737 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.555627  2490 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.556170  2490 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.557209 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:17.558280  2490 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.489841  2709 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.581794 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.582612 31141 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
W20260709 12:25:17.583019 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 98712.1us
W20260709 12:25:17.583719 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 146935us
I20260709 12:25:17.589710 31068 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:17.567864  2743 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.592751 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.594537 31141 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:17.569298  2609 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.569554  2578 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:17.604646 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.596035 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:17.592916 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:17.606582 31697 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
W20260709 12:25:17.608343 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.610929 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:17.612846 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.584781  2649 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.618042 31141 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:17.617965 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:17.558780  2528 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:25:17.596276  2741 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:17.682150 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 241847us
I20260709 12:25:17.614282  2625 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.736290 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:17.638543  2710 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.604887  2578 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.637573  2721 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:17.612112  2698 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:17.549378  2744 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.611219  2609 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.745446 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 637) took 1409 ms. Trace:
I20260709 12:25:17.559048  2728 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.767136 31140 raft_consensus.cc:2368] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8.
I20260709 12:25:17.765328 31144 rpcz_store.cc:276] 0709 12:25:16.336059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.428855 (+ 92796us) service_pool.cc:224] Handling call
0709 12:25:16.437312 (+  8457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.442020 (+  4708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.442049 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:16.442435 (+   386us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.802865 (+360430us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:16.802942 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.660453 (+857511us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b54003f4dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:17.675515 (+ 15062us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:17.675572 (+    57us) raft_consensus.cc:1685] finished
0709 12:25:17.675589 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.745375 (+ 69786us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5749,"spinlock_wait_cycles":9817984,"thread_start_us":329694,"threads_started":1,"wal-append.queue_time_us":361138}
I20260709 12:25:17.619475  2647 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 12:25:17.769822 31135 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:17.621903  2602 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 12:25:17.773360 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:17.675643  2553 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:17.763288  2679 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.619s	user 0.007s	sys 0.006s
W20260709 12:25:17.771760 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.749681  2740 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:17.784981  2751 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:25:17.785306  2737 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.764810  2698 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.790587  2698 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:17.790337  2757 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:25:17.745963  2732 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.763366  2742 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:17.795055  2742 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.795539  2742 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:17.769418  2578 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:17.763692  2721 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.766796  2756 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:17.769268  2706 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:17.816018 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.817144  2728 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:17.817883 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:17.818612 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:17.819314 31690 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:25:17.819538  2744 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.821483  2737 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:17.822525  2588 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.540s	user 0.003s	sys 0.007s
W20260709 12:25:17.823549  2654 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 12:25:17.824146  2654 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.738633  2741 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.828918 31692 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:25:17.830165 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.506s (SENT) [suppressed 7 similar messages]
I20260709 12:25:17.791679  2686 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.840426 31695 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:17.841163 31697 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:17.844512 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:17.776995  2553 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:17.873477 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:17.869719  2758 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:17.874125 31065 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:17.809765  2721 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:17.870824  2654 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:17.872377  2625 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:17.828985  2698 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.870369  2741 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.815894  2709 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:17.812577  2756 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:17.891501 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:17.791335  2726 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.807660  2732 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.892133 31059 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.901291  2732 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:17.893801  2756 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:17.902405  2553 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.903352  2553 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.909019  2698 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.912147  2578 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.913723  2514 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:17.917016 31065 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:17.918262 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:17.923611 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:17.924494 31068 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.927421  2726 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.928005  2726 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:17.928902  2396 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:17.929100  2654 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:17.929364  2396 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:17.929595  2654 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:17.930501  2654 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.931468  2654 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:17.935003 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.935629 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.935426  2396 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.936793  2396 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:17.937826  2709 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.938342  2709 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.938849  2514 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:17.939622  2514 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:17.940411  2709 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.950791 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:17.991992  2731 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:17.992111 31065 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.992597  2731 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.807757  2740 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.955665  2609 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:17.961040  2584 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.595s	user 0.008s	sys 0.005s
I20260709 12:25:17.995536 31139 raft_consensus.cc:2410] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:17.995918  2740 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:17.996717  2436 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:17.963619  2729 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:17.940793  2514 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.965094 31142 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:25:17.998171  2729 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:17.999680  2729 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:17.962754  2696 consensus_meta.cc:287] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.001s
I20260709 12:25:18.001252  2696 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:17.976473 31135 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:17.977401 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.978204 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.982893 31696 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:17.990547 31693 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:17.984138 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:17.873275  2686 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:18.013865 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.022351 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.032891 31058 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:18.012320 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:17.975692 30958 connection.cc:256] RPC call timeout handler was delayed by -1.0084204700001465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
I20260709 12:25:18.012832 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.025461 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:18.034925 31060 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:18.038609 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:18.039373  2278 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:18.039773  2278 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.036311 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:18.039963  2528 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:18.040180  2278 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:18.040521  2528 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.040016 31060 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:25:18.043087 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:18.047609 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:17.979825  2289 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:18.049564  2289 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.050323  2528 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.050822  1995 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:17.946938  2597 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.051326  1995 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:17.990548  2685 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:18.053725 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:18.048781 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:17.977054  2485 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:18.047618  2736 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.057828  2609 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:18.051756  2289 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.070022  2731 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.070370  2485 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.071364 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.072614  2485 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:18.072717 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:18.072973  2278 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.074268  2599 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:18.075024  2599 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:18.075207  2278 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.075449  2597 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.016752  2703 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:18.022004  2729 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:18.076171  2703 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:18.077025  2703 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.060921 31059 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:18.079001 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:18.079588 31059 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:18.079800  2686 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.079864 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.080741 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:18.081379 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.058372 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.061532 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:18.065464  2756 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.075517  2599 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:18.083195  2599 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.083933  2756 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:18.084583 31063 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:18.066058  2730 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:18.066764  1995 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.066531  2736 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.058969  2685 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:18.035275  2584 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.075901  2737 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:18.098150 31695 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:18.078032  2703 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:18.082074 30986 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:18.065636  2528 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:18.085318 31068 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:18.087107 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.106173 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:18.087414 31065 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
W20260709 12:25:18.088102 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.058682  2696 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.089900 30989 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 12:25:18.090911 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.107527 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:17.988744  2752 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.250s	user 0.000s	sys 0.000s
I20260709 12:25:18.116215 31059 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
W20260709 12:25:18.089990  2630 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.000s
I20260709 12:25:18.052958  2740 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:18.109225 30985 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:25:18.110476 30988 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:25:18.106420 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.110872 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.094029 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:17.896462 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 52088.6us
I20260709 12:25:18.123224 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:18.126878 31058 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.126753 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:18.130247 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.132822 31136 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.136516 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.136510 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:18.137564 31065 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:18.139236 31063 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:25:18.139698 31063 pending_rounds.cc:85] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 7
I20260709 12:25:18.139964 31063 pending_rounds.cc:107] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:25:18.140241 31063 raft_consensus.cc:2889] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:18.117913 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:18.125558 31068 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:18.108850  2528 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.142863  2528 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:18.143958  2736 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.141871 31059 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.096976  2730 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:18.019351  2742 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.146034  2730 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.147796  2742 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.148432  2278 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:18.148968  2278 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:18.149880 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 43790.1us
I20260709 12:25:18.149785  2278 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.150851 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.137317 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.155086 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.155843 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.156601  2396 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:18.096746  2566 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:18.102681  2507 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:18.158414 31144 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:18.167217 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.158921 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.160024 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:18.167094 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 336959us
I20260709 12:25:18.161900 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.169284 31068 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:18.170076 31138 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:25:18.111869  2726 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:18.189379 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:18.157104  2396 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.190197 31693 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:18.173586 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.175794 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:18.177495 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:18.177589 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:18.180292 31059 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:18.182586 31690 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:18.177909 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:18.190505 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:18.191684 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:18.194238 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:18.194319 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:18.211726 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.119094  2737 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:18.108587  2584 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.141834  2630 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.164762 31136 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:18.219835 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:18.358820 31062 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.017s	sys 0.067s
W20260709 12:25:18.445127 31062 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.020s	sys 0.070s
W20260709 12:25:18.310866 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.446376 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:18.264730 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.445034 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.342355 31058 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:18.447227 30988 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:18.250528 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:18.249822 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 89866.1us
W20260709 12:25:18.385002 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.331036 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:18.358349 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.447434 31141 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:18.451344 30991 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 13: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:18.451274 31693 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:18.177348  2742 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:18.280256 30989 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:25:18.455711 31064 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.052s	sys 0.103s
I20260709 12:25:18.452591 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.456530 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.454594 31139 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:18.445200 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:18.457475 31064 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.052s	sys 0.103s
W20260709 12:25:18.446565 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.459227 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.459662 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.458057 31697 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:18.447324 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:18.167040  2566 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.478260 31033 leader_election.cc:400] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8.
I20260709 12:25:18.478619 31033 leader_election.cc:403] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:18.480759 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:18.487102 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.172089  2696 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:18.190580  2736 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:18.217268  2396 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:18.498682 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:18.499035 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:18.195180  2726 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.502031 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:18.394359  2597 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:18.249522  2604 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:18.220264  2436 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:18.268610  2507 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:18.506732 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.217519  2732 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.501405 31142 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 7.
I20260709 12:25:18.508512  2686 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:18.431099  2485 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:18.529184 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:18.322428  2609 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:18.454504  1995 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:18.136607  2740 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.483135  2742 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.486057  2566 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:18.489785  2553 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8.
I20260709 12:25:18.517304 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.490051  2702 raft_consensus.cc:2764] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:25:18.557749 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.005s	sys 0.011s
I20260709 12:25:18.307777  2625 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.501798  2736 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.527005  2578 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.526650  2436 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:18.514144  2726 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:18.532024 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:18.529305  2485 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.516369  2748 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:18.579181  2748 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:18.580418  2485 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.580071  2748 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:18.521525  2241 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.009s	sys 0.005s
W20260709 12:25:18.582428  2241 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.010s	sys 0.005s
I20260709 12:25:18.583285  2528 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:18.585240 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.586108 30992 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:18.586464 30992 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:18.560678  2609 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:18.505856  2696 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.558429 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.005s	sys 0.011s
I20260709 12:25:18.587087 30992 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.531265  2732 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:18.533257  2514 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.003s	sys 0.029s
I20260709 12:25:18.269668  2685 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.518136  2596 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
W20260709 12:25:18.560305 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.002s	sys 0.014s
I20260709 12:25:18.520004  2597 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.530325 31068 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:18.527256  2686 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:18.566937 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:18.511500  2756 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:25:18.628975 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:18.543040  2604 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:18.557439  2742 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:18.564491  2736 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.571599  2578 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:18.554114  1995 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.679432 31139 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:25:18.510684  2746 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.003s	sys 0.018s
I20260709 12:25:18.596535 31138 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:18.597133 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:18.558813 30983 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.010s	sys 0.006s
I20260709 12:25:18.597961 30992 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:18.680879 30983 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.010s	sys 0.006s
W20260709 12:25:18.598127 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.602548  2528 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:18.604293  2588 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.008s	sys 0.018s
I20260709 12:25:18.604449 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:18.685074 31059 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:18.610239 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.002s	sys 0.015s
I20260709 12:25:18.585860  2702 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.612323 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.690330 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.690770 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.565320  2625 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.692023 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.614882 31698 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:25:18.612608 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:18.633636  2597 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.634483 31137 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:18.675871  2685 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:18.638691  2609 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:18.651710  2723 consensus_queue.cc:1175] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a517aadb0ae04b89a13779d7898dfa0c" responder_term: 9 status { last_received { term: 9 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 9 index: 8 } }
I20260709 12:25:18.548839  2737 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.659008  2436 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.661585  2732 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.667112  2740 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.668098  2630 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.668896  2584 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:18.609244  2514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.004s	sys 0.029s
I20260709 12:25:18.697302 31063 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:18.699785 31067 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
W20260709 12:25:18.703680 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.630990  2686 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:18.748915  2604 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.708843  2702 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.754262  2702 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:18.741340  2721 consensus_meta.cc:287] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.844s	user 0.006s	sys 0.004s
I20260709 12:25:18.756834  2721 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.758885  2721 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.635046 30989 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 13: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:18.707944 30992 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:25:18.683565  2746 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.003s	sys 0.018s
I20260709 12:25:18.598834  2696 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:18.685720  2654 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.754s	user 0.011s	sys 0.000s
I20260709 12:25:18.711359 31139 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 12:25:18.706055 30955 connection.cc:256] RPC call timeout handler was delayed by -1.842337850000149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 12:25:18.684847  2588 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.008s	sys 0.018s
I20260709 12:25:18.709388 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.727658 31059 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:18.595579  2507 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.679798  2485 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:25:18.748978  2514 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.759058  2566 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.760473 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.686103  2396 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.764703 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.829511 31063 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:18.830646 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:18.744292  2703 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.000s
I20260709 12:25:18.741737  2528 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.791225 30955 connection.cc:256] RPC call timeout handler was delayed by -1.6376095050004551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
W20260709 12:25:18.839598 30955 connection.cc:256] RPC call timeout handler was delayed by -1.6071158170002491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 12:25:18.839886 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:18.840178 30955 connection.cc:256] RPC call timeout handler was delayed by -1.4910724040000787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 12:25:18.840585 30955 connection.cc:256] RPC call timeout handler was delayed by -1.1593864379997285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
I20260709 12:25:18.793183 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.841002 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.805697 31692 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
W20260709 12:25:18.809459 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 359216us
I20260709 12:25:18.728636  2597 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:18.834820 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:18.799010  2747 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:18.789882  2702 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.841747 31063 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:18.842199 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:18.873346  2490 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:18.873863  2490 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:18.874775  2490 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:18.878417 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.836195 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 383643us
W20260709 12:25:18.864117 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.870121 31136 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:18.884495 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:18.914350  2767 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.915022  2767 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.917495  2767 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.919574  2696 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.926491  2750 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.926981  2750 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.927997  2746 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.928469  2746 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.928930  2750 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:18.930050  2625 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.930428  2746 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.931185  2746 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.931658  2396 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.931607  2746 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.933457  1995 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.933570  2746 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:18.934239  2746 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:18.934691  2746 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.936470  2566 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:18.944690 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.947031 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:18.921969  2696 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.948359  1995 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.964948 31060 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.050s	sys 0.055s
W20260709 12:25:18.965291 31060 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.050s	sys 0.055s
I20260709 12:25:18.835736  2514 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:18.974785 31061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.031s	sys 0.059s
W20260709 12:25:18.975216 31061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.031s	sys 0.059s
W20260709 12:25:18.975313 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.867501  2528 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.936468  2746 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:18.956904  2396 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:18.978650 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:18.868798  2686 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.989120 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:18.870440  2597 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:18.893540  2702 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:18.997253 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:19.007395 31068 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:19.009181 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:19.014986 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:19.015395 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:19.015791 31059 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:18.997337 31140 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.010s	sys 0.018s
W20260709 12:25:19.003584 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.020848 31140 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.010s	sys 0.019s
W20260709 12:25:19.004747 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.964615  1995 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:19.026912 31067 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:19.021762 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:19.029887 31063 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:25:19.031625 31063 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:25:19.032012 31063 pending_rounds.cc:85] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 9
I20260709 12:25:19.032263 31063 pending_rounds.cc:107] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:19.032526 31063 raft_consensus.cc:2889] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:19.068462 31064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.027s	sys 0.022s
W20260709 12:25:19.072397 31064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.027s	sys 0.022s
I20260709 12:25:19.073279 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2410) took 1302 ms. Trace:
I20260709 12:25:19.028949 31106 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:19.078774 31059 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:19.081199 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:19.083704 31060 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:25:19.086310 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57455.9us
I20260709 12:25:19.087729 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:19.013844  2746 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:19.092051 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:19.073686 31064 rpcz_store.cc:276] 0709 12:25:17.770721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.770868 (+   147us) service_pool.cc:224] Handling call
0709 12:25:17.771186 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.771384 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:18.482352 (+710968us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.072909 (+590557us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.072957 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:19.072997 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.073120 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.073232 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60278,"prepare.queue_time_us":365330,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":17141,"thread_start_us":1257577,"threads_started":2}
W20260709 12:25:19.052081  2708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.012s	sys 0.026s
I20260709 12:25:19.050448  2584 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:19.051271  2740 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:19.024024  2729 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.966800  2696 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:19.121335 31065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.013s	sys 0.011s
W20260709 12:25:19.121768 31065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.013s	sys 0.011s
I20260709 12:25:19.070366  2578 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:19.123673 30986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.011s	sys 0.024s
W20260709 12:25:19.119424 31664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.009s	sys 0.026s
W20260709 12:25:19.162359 31664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.010s	sys 0.026s
W20260709 12:25:19.162354 30986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.013s	sys 0.025s
I20260709 12:25:19.163815 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:19.127597 31064 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:19.165340 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2114) took 1059 ms. Trace:
W20260709 12:25:19.119333  2708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.013s	sys 0.028s
I20260709 12:25:19.051434  2721 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:25:19.035598  2396 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
W20260709 12:25:19.134024  2278 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.000s	sys 0.036s
I20260709 12:25:19.117136  2723 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:19.180157 30958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.001s	sys 0.025s
W20260709 12:25:19.180490 30958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.001s	sys 0.025s
W20260709 12:25:19.180742 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 551019us
I20260709 12:25:19.183840 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:19.115353 31062 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:19.128170  2740 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:19.182301  2599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.096s	user 0.017s	sys 0.029s
W20260709 12:25:19.197031  2599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.111s	user 0.017s	sys 0.029s
I20260709 12:25:19.158624  2732 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:19.122380  2729 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:19.204741  2729 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:19.205646  2729 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:19.192394  2726 consensus_meta.cc:287] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.000s
I20260709 12:25:19.160813 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:19.120853  2746 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:19.158236 31058 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.007s	sys 0.030s
I20260709 12:25:19.121335  2630 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.130156  2756 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:19.214349 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.193435  2278 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.000s	sys 0.036s
W20260709 12:25:19.220300 31058 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.008s	sys 0.030s
W20260709 12:25:19.222661 31135 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.010s	sys 0.027s
W20260709 12:25:19.292440 31135 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.010s	sys 0.028s
I20260709 12:25:19.213049  2708 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:19.270023 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.009s	sys 0.022s
I20260709 12:25:19.193168  2723 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:19.200605  2241 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.018s	sys 0.010s
W20260709 12:25:19.286406 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.166010 30986 rpcz_store.cc:276] 0709 12:25:18.105997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.106887 (+   890us) service_pool.cc:224] Handling call
0709 12:25:18.128261 (+ 21374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.128476 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:18.524332 (+395856us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.164632 (+640300us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.164825 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.164887 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:19.164955 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.165115 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.165268 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":657561,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":1799,"raft.queue_time_us":103,"thread_start_us":1005702,"threads_started":2}
W20260709 12:25:19.297358 31107 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.016s	sys 0.019s
W20260709 12:25:19.320340 31107 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.017s	sys 0.019s
W20260709 12:25:19.299271 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.010s	sys 0.022s
I20260709 12:25:19.206168  2729 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:19.315560 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:19.326081 31062 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:19.218170  2746 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:19.329527 31108 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.004s	sys 0.037s
W20260709 12:25:19.334170 31108 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.004s	sys 0.037s
I20260709 12:25:19.281875  2778 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:19.339087 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:19.339394 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:19.347010 31139 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:25:19.325244  2241 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.018s	sys 0.011s
W20260709 12:25:19.355629 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.345166  2748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.015s	sys 0.029s
I20260709 12:25:19.215750  2732 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:19.368813 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.336117  2604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.006s	sys 0.020s
I20260709 12:25:19.325754  2723 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.377619 31142 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:25:19.347589  2742 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.000s
I20260709 12:25:19.345844  2778 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:19.270300  2726 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:19.329797  2528 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:19.398002 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.016s	sys 0.040s
I20260709 12:25:19.349543  2746 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.289824  2630 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:19.296041  2756 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:19.409242  2604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.006s	sys 0.020s
W20260709 12:25:19.375900  2748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.015s	sys 0.030s
W20260709 12:25:19.417548 31663 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.027s	sys 0.038s
W20260709 12:25:19.435417 31663 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.027s	sys 0.038s
W20260709 12:25:19.384366  2737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.026s	sys 0.035s
I20260709 12:25:19.375360  2514 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:19.439459 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:19.383783  2241 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.399466  2686 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:19.434182 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.421113 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.016s	sys 0.040s
W20260709 12:25:19.418836  2588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.014s	sys 0.016s
I20260709 12:25:19.326927  2578 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:19.380273  2597 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:19.422765  1995 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:19.418031  2784 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:19.425902  2774 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 12:25:19.440250 31139 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:19.451438 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:19.443763  2737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.026s	sys 0.036s
W20260709 12:25:19.437799  2507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.023s	sys 0.011s
I20260709 12:25:19.386940  2740 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:19.449054  2588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.014s	sys 0.017s
I20260709 12:25:19.473093 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:19.444591  2131 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.476351 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:19.479530 31062 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:19.420637  2742 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:19.493297 31067 raft_consensus.cc:2410] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:25:19.437939 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.010s	sys 0.021s
W20260709 12:25:19.507283 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 300764us
W20260709 12:25:19.989342 31136 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.000s	sys 0.001s
W20260709 12:25:19.968782 31109 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.027s	sys 0.052s
W20260709 12:25:19.989951 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.010s	sys 0.022s
W20260709 12:25:19.990175 31109 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.032s	sys 0.053s
I20260709 12:25:19.450412  2241 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:19.990826 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 801873us
W20260709 12:25:19.968101 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.014s	sys 0.013s
W20260709 12:25:19.991523 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.014s	sys 0.014s
W20260709 12:25:19.968132 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.008s	sys 0.007s
W20260709 12:25:19.993217 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.009s	sys 0.007s
W20260709 12:25:19.993925 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 810135us
W20260709 12:25:19.989926 31136 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.000s	sys 0.002s
I20260709 12:25:19.495815  2131 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:19.996666 31141 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:19.981513 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 668) took 1884 ms. Trace:
I20260709 12:25:19.997298 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:19.990554 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 1.14928e+06us
W20260709 12:25:19.978758 30958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.013s	sys 0.009s
I20260709 12:25:19.999296 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:19.997638 31135 rpcz_store.cc:276] 0709 12:25:18.096364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.096510 (+   146us) service_pool.cc:224] Handling call
0709 12:25:18.102657 (+  6147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.565842 (+463185us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:18.566150 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.296630 (+730480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.296699 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:19.296759 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.980608 (+683849us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b5400062554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:19.980724 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.981117 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54348,"prepare.queue_time_us":591,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":4089,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8650,"spinlock_wait_cycles":1064625920,"thread_start_us":726398,"threads_started":1,"wal-append.queue_time_us":901520}
W20260709 12:25:19.480753  2490 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.028s	sys 0.060s
W20260709 12:25:19.505960  2609 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.005s	sys 0.047s
W20260709 12:25:20.002828  2490 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.028s	sys 0.076s
W20260709 12:25:20.002913  2609 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.015s	sys 0.063s
W20260709 12:25:19.476549  2507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.023s	sys 0.011s
I20260709 12:25:20.001921  2241 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:20.005475  2584 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.449102  2729 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:19.476953  2784 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:19.453899  2796 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 12:25:19.999845 30958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.014s	sys 0.010s
W20260709 12:25:19.983436 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.053936 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 871971us
W20260709 12:25:20.054380 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.055562  2784 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:20.055701 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:19.973593 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2425) took 1464 ms. Trace:
W20260709 12:25:19.985172 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.029449 31141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2179) took 1704 ms (client timeout 1713 ms). Trace:
I20260709 12:25:19.491541  2790 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:20.030562 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.506459  2798 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.537s
I20260709 12:25:20.029465 31695 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 11. Ops: []
I20260709 12:25:19.500145  2778 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.033627  2241 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:20.034137  2748 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:25:20.043959  2584 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:19.638839  2783 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
W20260709 12:25:20.067485 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.078135  2241 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:20.078711  2241 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:20.056842 31141 rpcz_store.cc:269] 0709 12:25:18.324619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.483911 (+159292us) service_pool.cc:224] Handling call
0709 12:25:20.027742 (+1543831us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540024a254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:20.029334 (+  1592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2389582080}
I20260709 12:25:20.087662  2131 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:20.089521  2485 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.090371  2485 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:20.096694  2736 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.399s	user 0.012s	sys 0.000s
I20260709 12:25:20.097218  2736 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.100610  2732 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:20.106609 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 116913us
I20260709 12:25:19.999867 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:19.785187  2768 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
I20260709 12:25:20.037547  2726 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:19.639822  2793 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 12:25:19.778129  2781 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.612s	user 0.003s	sys 0.000s
W20260709 12:25:19.780627  2788 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.400s	user 0.000s	sys 0.003s
W20260709 12:25:19.689670  2795 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:25:19.483973  2632 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.572s
I20260709 12:25:19.793767  2746 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.065438  2790 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:20.066195  2630 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.107872 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:19.703143  2805 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.361s
W20260709 12:25:19.639382  2782 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 12:25:19.323505 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:19.637619  2785 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:25:20.133008  2790 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.139475  2790 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.140545 31063 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 12:25:20.140908 31063 pending_rounds.cc:85] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 8
I20260709 12:25:20.141147 31063 pending_rounds.cc:107] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 7.9
W20260709 12:25:19.637569  2791 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 12:25:19.688148  2794 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:25:20.130260 31143 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:20.141462 31063 raft_consensus.cc:2889] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:20.144198  2632 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:20.144841 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:20.145334 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 952) took 2591 ms. Trace:
I20260709 12:25:20.137593  2756 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:19.774106  2766 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.152345  2632 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:20.152586 31061 rpcz_store.cc:276] 0709 12:25:17.553697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.556392 (+  2695us) service_pool.cc:224] Handling call
0709 12:25:17.556707 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.792237 (+235530us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:18.106721 (+314484us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.975767 (+869046us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.975838 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:18.975897 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.779959 (+804062us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:20.144917 (+364958us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54001b9c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:20.144985 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.145138 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":476640,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":1615,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":19303,"spinlock_wait_cycles":570442880,"thread_start_us":1173303,"threads_started":2,"wal-append.queue_time_us":1191741}
W20260709 12:25:20.155884 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 156275us
I20260709 12:25:20.057655 31058 rpcz_store.cc:276] 0709 12:25:18.509276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.509468 (+   192us) service_pool.cc:224] Handling call
0709 12:25:18.509900 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.291931 (+782031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.291976 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.292036 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:19.292083 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.972707 (+680624us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b54000c6a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:19.972856 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.973293 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11127,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":121863552,"thread_start_us":728516,"threads_started":1}
I20260709 12:25:20.158878  2630 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:20.159416  2630 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:20.159921  2630 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:20.160096  2632 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.161393  2737 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:20.161916  2632 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.162722  2779 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:20.165871  2779 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.166313  2779 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:20.168193 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 969) took 2098 ms. Trace:
I20260709 12:25:20.168613 31065 rpcz_store.cc:276] 0709 12:25:18.069399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.139176 (+ 69777us) service_pool.cc:224] Handling call
0709 12:25:18.139583 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.161142 (+ 21559us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:18.548674 (+387532us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.160627 (+611953us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.169154 (+  8527us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:19.169733 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 12:25:19.169872 (+   139us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:19.169910 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.169974 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:20.167725 (+997751us) raft_consensus.cc:1685] finished
0709 12:25:20.167774 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.168027 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52211,"prepare.queue_time_us":387632,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":148594,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":22963,"thread_start_us":831947,"threads_started":2,"wal-append.queue_time_us":917435}
I20260709 12:25:20.173882 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.174971 31064 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:20.176676 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2355) took 3226 ms. Trace:
I20260709 12:25:20.178421 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:20.177101 30983 rpcz_store.cc:276] 0709 12:25:16.950039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.114940 (+164901us) service_pool.cc:224] Handling call
0709 12:25:17.116184 (+  1244us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:17.985328 (+869144us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b180020ecc0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb88cbfb 00007f2eeb89764b
0709 12:25:17.985460 (+   132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.985486 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:17.986037 (+   551us) log.cc:844] Serialized 52 byte log entry
0709 12:25:18.681348 (+695311us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:18.996365 (+315017us) log.cc:844] Serialized 14 byte log entry
0709 12:25:19.003545 (+  7180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.003624 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.638819 (+635195us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b180020ecc0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:19.784486 (+145667us) spinlock_profiling.cc:243] Waited 8.16 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:20.176206 (+391720us) raft_consensus.cc:1685] finished
0709 12:25:20.176242 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.176441 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":773896,"prepare.queue_time_us":129208,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":40980,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":20189,"spinlock_wait_cycles":166048768,"thread_start_us":1028595,"threads_started":3,"wal-append.queue_time_us":883188}
I20260709 12:25:20.179615  2784 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:20.180019  2784 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:20.180454  2784 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.162077  2737 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:20.183682  2737 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:20.187264 31143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2174) took 1996 ms (client timeout 2233 ms). Trace:
W20260709 12:25:20.187772 31143 rpcz_store.cc:269] 0709 12:25:18.191189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.191424 (+   235us) service_pool.cc:224] Handling call
0709 12:25:20.186998 (+1995574us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b540006efd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:20.187189 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4367182208}
I20260709 12:25:20.197196 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.200145  2732 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.200996  2732 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.199348 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.203514 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.204516 31060 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:20.204890 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.206702 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.207861  2696 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:20.209457 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 210504us
I20260709 12:25:20.214673  2584 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.215391  2584 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.216171  2584 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.216562  2742 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:20.220544  2746 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:20.225884  2767 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:20.226732  2767 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.227283  2584 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.229242  2767 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.231573  2822 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.232311  2822 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.234059  2778 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:20.235718  2702 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:20.229998  2767 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:20.241304 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.242929  2766 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.155864  2485 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.248471  2485 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:20.248997  2485 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:20.250487  2485 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.272989  2732 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.276536  2822 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:20.272019 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49745.5us
W20260709 12:25:20.282517  2685 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.554s	user 0.014s	sys 0.007s
I20260709 12:25:20.282886 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.286862 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:20.291640 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 79565.7us
I20260709 12:25:20.278292  2766 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:20.296690 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 189102us
I20260709 12:25:20.296690 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.298796 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.293066 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.296598  1995 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.305555 30989 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:20.305752  1995 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.308177 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:20.305811 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 127627us
I20260709 12:25:20.299398 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.329288 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.332916 30988 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:20.355417 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:20.358048  2396 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:20.358712  2396 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:20.360018  2396 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.360354  2584 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:20.361920  2584 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:20.362319  2584 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:20.382692  2736 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:20.383209 31033 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:20.383664 31033 leader_election.cc:403] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.384691 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:20.386428  2702 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:20.387431  2702 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:20.387887  2702 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.388314  2702 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:20.298015  2732 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:20.391140  2732 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:20.394650  2708 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.069s	user 0.015s	sys 0.002s
I20260709 12:25:20.395239  2708 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.396546 30991 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:20.400417  2630 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.404305  2784 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.406260  2685 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.406805 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 995) took 2247 ms. Trace:
I20260709 12:25:20.407176  2685 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.410384  2742 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.411000  2742 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.412480 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.412894  2789 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:20.413839  2789 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.385869 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:20.421517 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 131474us
I20260709 12:25:20.393940  2732 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.423295  2726 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:20.423832 31106 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:20.424817 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 66589.2us
I20260709 12:25:20.427181  2732 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:20.427816  2732 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:20.428437  2732 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.430415  2630 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:20.431710  2584 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.407325 31695 rpcz_store.cc:276] 0709 12:25:18.159404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.160461 (+  1057us) service_pool.cc:224] Handling call
0709 12:25:18.217818 (+ 57357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.064509 (+1846691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.406381 (+341872us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b540055c654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:20.406668 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4115427072}
I20260709 12:25:20.471697  2766 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:20.629860 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.628170 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.633451 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 103612us
W20260709 12:25:20.635932 30958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.001s	sys 0.022s
W20260709 12:25:20.636323 30958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.001s	sys 0.022s
W20260709 12:25:20.636889 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 581361us
I20260709 12:25:20.637558 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:20.494149  2790 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:20.638669  2485 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:20.639379 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 244397us
W20260709 12:25:20.643469 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.505777 30991 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:20.660944  2742 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.661676 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.654948  2485 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:20.663967  2485 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:20.665483 30887 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 313) took 24772 ms (client timeout 15000 ms). Trace:
W20260709 12:25:20.666785 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.667749 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:20.536562  2723 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.114s	user 0.008s	sys 0.007s
I20260709 12:25:20.669135 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:20.670857 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.618114  2685 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.546954  2816 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:20.499400  1995 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.667675  2784 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.679970 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.681506 31060 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:20.682648 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:20.683285 31060 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.683423 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:20.684187 31067 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:20.632078  2241 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.688277  2241 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:20.688361  2584 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.696497 31067 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
W20260709 12:25:20.698354 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.703184  2796 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 12:25:20.704607 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.531227  2779 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:20.665901 30887 rpcz_store.cc:269] 0709 12:24:55.892742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.892941 (+   199us) service_pool.cc:224] Handling call
0709 12:25:20.665228 (+24772287us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:15.780341 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:15.780906 (+   565us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:15.780939 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:25:15.780958 (+    19us) tablet.cc:662] Decoding operations
0709 12:25:15.783010 (+  2052us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:15.783112 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:15.783157 (+    45us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:25:15.785063 (+  1906us) tablet.cc:701] Row locks acquired
0709 12:25:15.785082 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:25:15.785270 (+   188us) write_op.cc:270] Start()
0709 12:25:15.785430 (+   160us) write_op.cc:276] Timestamp: P: 1783599915785207 usec, L: 0
0709 12:25:15.785456 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:25:15.932689 (+147233us) log.cc:844] Serialized 7481 byte log entry
0709 12:25:16.477620 (+544931us) op_driver.cc:464] REPLICATION: finished
0709 12:25:16.680320 (+202700us) write_op.cc:301] APPLY: starting
0709 12:25:16.680414 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:16.682116 (+  1702us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:16.686548 (+  4432us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:16.856203 (+169655us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:16.856520 (+   317us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=38
0709 12:25:16.856567 (+    47us) write_op.cc:312] APPLY: finished
0709 12:25:16.856916 (+   349us) log.cc:844] Serialized 132 byte log entry
0709 12:25:16.857098 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:16.857272 (+   174us) write_op.cc:454] Released schema lock
0709 12:25:16.857327 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:16.858480 (+  1153us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:16.858507 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1031873,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":199549,"apply.run_cpu_time_us":6542,"apply.run_wall_time_us":178126,"num_ops":19,"prepare.queue_time_us":140999,"prepare.run_cpu_time_us":10540,"prepare.run_wall_time_us":226056,"raft.queue_time_us":429541,"raft.run_cpu_time_us":19596,"raft.run_wall_time_us":181846,"replication_time_us":670236,"spinlock_wait_cycles":67328,"thread_start_us":603350,"threads_started":4,"wal-append.queue_time_us":72263,"wal-append.run_cpu_time_us":88820,"wal-append.run_wall_time_us":1853893}]]}
I20260709 12:25:20.675928  2816 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:20.698648  2822 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:20.706758 31067 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:20.712666 31060 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:20.706957 30955 leader_election.cc:400] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10.
I20260709 12:25:20.713915 30955 leader_election.cc:403] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.679065  2789 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.714670 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.719897 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:20.721048  2632 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:20.721684 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:20.725474  2833 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.087s	user 0.000s	sys 0.007s
I20260709 12:25:20.726306 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.715020 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.721944 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:20.712178  2822 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:20.743765  2831 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:25:20.744666 31695 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:20.681136  2723 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.736590  2784 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.751801 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.746201  2632 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.754014 31067 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:20.744231  2796 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10.
I20260709 12:25:20.757423  2784 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.758812 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.759697  2784 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:20.765178 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 88827.2us
W20260709 12:25:20.773878  2836 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.136s	user 0.000s	sys 0.002s
I20260709 12:25:20.775612 31143 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 12:25:20.777333  2834 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
I20260709 12:25:20.777549  2767 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:20.777853 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.761826  2702 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.777964  2767 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:20.781080  2702 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:20.781687  2767 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.782969 31067 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:25:20.783263  2829 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:25:20.783623  2767 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:20.796329 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.799928 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:20.800645 31143 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:20.804270 31136 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:20.804852  2780 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:20.806447 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:20.806545  2822 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.807245 31137 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:20.808082  2822 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:20.808480 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:20.809254 31142 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:20.810236 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:20.811019 31141 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:20.811344 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.811919 30988 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:20.812036 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:20.814520 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.815281 31695 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:20.820434 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:20.820919  2823 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:20.821156  2816 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.822777  2816 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.825687 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:20.827422  2823 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:20.820645  2832 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:20.829819  2823 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:20.831502  2823 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.801931 30886 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 10, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 10 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.835747 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.836599 30991 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:20.837353 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.837951 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.838539 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.838621 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.837973 30987 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:25:20.839298 30983 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:20.839565 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.839751 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.835986 30886 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 10, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:25:20.840696 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 118947us
I20260709 12:25:20.842756 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.843014 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:20.843743 30992 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:20.844928  2792 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.845480  2726 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:20.847110  2702 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:20.821074 31058 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:20.848328 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:20.841250 30886 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:20.875648 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:20.879992 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.886054 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:20.887338 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 85599.7us
I20260709 12:25:20.884802 30886 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:20.888005  2779 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:20.858632  2702 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.899529 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.899618 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.856278  2737 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:25:20.904507 31695 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:20.898919 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:20.907599 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.904126 30886 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:20.912835 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.913022 30886 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.904596  2584 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:20.917698  2820 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:25:20.911646 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.910337  2396 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:20.914376  2742 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:20.927711 31062 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:20.923171 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.928853 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:20.928678 31695 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:25:20.936707 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.938125  2740 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:20.938640 30990 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:20.938822  2740 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:20.939493 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.940179 30989 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:20.941748  2740 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:20.943461  2584 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:20.944700  2584 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.945587  2632 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:20.927564  2742 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:20.963537 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:20.965729 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.977283 30983 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:20.980058  2784 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.980616  2784 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:20.981236 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.981828 30988 raft_consensus.cc:2376] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14. Re-sending same reply.
I20260709 12:25:20.983426  2784 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:20.918831  1995 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:25:20.987641 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.988816 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.989711  2702 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:20.990845 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.992506 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:20.994463 30992 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:20.995364 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.996151 31695 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:20.996016 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.983600 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:20.996927 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.997373 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.998365 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:20.998572 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:20.999609 31698 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:20.972119 30886 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 10, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 10 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.998509  2805 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:21.007346 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45896.7us
I20260709 12:25:20.998018 31697 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 6.
I20260709 12:25:20.998791 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.013926 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.954401  2742 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:21.014606 30992 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:21.016876 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.024847 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:21.029362 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:21.013828  2732 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.004s
I20260709 12:25:21.036412 31064 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:21.030921 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.033308 30886 catalog_manager.cc:5697] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 7, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 7 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:21.007241  2820 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.023975 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.051160 31698 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:21.053462 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:20.975843  2740 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:25:21.058079 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.058945 30986 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:21.064333 31697 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:21.064210 30986 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:21.052393  2732 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:21.065435 30986 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:21.041718 31033 connection.cc:256] RPC call timeout handler was delayed by -1.2634621819997847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 12:25:21.067792 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.068555  2732 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:21.051563  2723 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:21.069784 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.030448  2846 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.070065 31033 connection.cc:256] RPC call timeout handler was delayed by -1.1581621820000692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
W20260709 12:25:21.070842 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0474621819998902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 12:25:21.070570  2846 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:21.072026 31033 connection.cc:256] RPC call timeout handler was delayed by -1.04370409799958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
I20260709 12:25:21.074129 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.077625 30986 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:21.074144 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 127153us
W20260709 12:25:21.080137 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.083460 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.087525  2798 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:25:21.088706  2798 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:21.095630  2798 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:21.096457  2798 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:21.097136 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:21.097714  2798 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.098349 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:21.099026 31143 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:21.100232 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:21.101011 31067 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:21.086941 31698 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
W20260709 12:25:21.110745  2842 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:25:21.111943 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.113665 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.114413 31690 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:21.077368  2396 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:21.119189 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:21.122845 31143 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
W20260709 12:25:21.126377  2853 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:25:21.126477 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.135988 30990 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:21.125810 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:21.127965 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:21.137460 31142 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:21.151518 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:21.290899 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.165342 31141 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.221789 31064 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:21.140527 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:21.150813 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 135860us
W20260709 12:25:21.276475 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 46132.7us
I20260709 12:25:21.343698 31690 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:21.319897 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.439010 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 95566.4us
W20260709 12:25:21.472002 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.472126 31058 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.601s	user 0.006s	sys 0.005s
I20260709 12:25:21.476629 31058 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:21.205811 31140 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:21.154409 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:21.473141 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.155925 31693 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:25:21.219451 31033 leader_election.cc:336] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.506s (SENT)
I20260709 12:25:21.570040 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:21.500667 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.573552 30990 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:21.314993  2702 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.227298 30886 catalog_manager.cc:5697] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 6, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 6 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:21.205467  2729 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:21.500610 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.521528 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:21.523301 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:21.544862 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:21.165462 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:21.497414 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.501302 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:21.206614  2856 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:25:21.562490 31067 pending_rounds.cc:85] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 7
I20260709 12:25:21.582652 31067 pending_rounds.cc:107] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:25:21.582973 31067 raft_consensus.cc:2889] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.546046 31135 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:21.564157 30985 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:21.564379 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:21.566401  2732 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.567714  2857 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:25:21.537196  1995 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.591722 30986 consensus_meta.cc:287] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.003s
I20260709 12:25:21.569312 31137 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:25:21.207376  2848 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:25:21.342523  2767 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.224s
I20260709 12:25:21.573490 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:21.585103 31141 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:21.586028  2729 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:21.604051 30955 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:21.604920  2729 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:21.605320  2798 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:21.586364 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 366925us
I20260709 12:25:21.587203 31143 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:21.584268 31139 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
W20260709 12:25:21.601068 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 79651.9us
I20260709 12:25:21.606081  2798 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:21.609791 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:21.610596  2798 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:21.611858 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:21.612545 31068 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.018s	sys 0.026s
I20260709 12:25:21.613161  2798 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:21.613291 31068 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.018s	sys 0.026s
I20260709 12:25:21.613711  2798 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:21.614122 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:21.614157  2798 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.615535 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:21.615736  2729 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:21.616430 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:21.617139  2729 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.618784  2729 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.622700 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:21.623677  2820 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:21.584834  2702 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:21.625216  2820 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:21.625691  2805 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:21.205312  2396 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:21.632606 31136 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.633582 30986 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:21.635069 30987 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:21.635452 30987 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:21.636257 30987 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:21.637444 30987 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.638471 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.600605  2732 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:21.641383  2732 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:21.569962  2796 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.004s	sys 0.011s
I20260709 12:25:21.642195 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:21.652940 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:21.654991 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:21.656574 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.648388 30886 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 5 to 7, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 7 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:21.658900 30987 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:21.590736  2784 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.655280 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:21.642421  1995 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:21.666687  1995 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:21.668804 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.670068 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.673137  2852 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:21.674000  2849 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
I20260709 12:25:21.674898 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.675861 30985 raft_consensus.cc:2410] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:21.676394 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.676972 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:21.667346  2732 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:21.677707 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.678390 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:21.678731 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:21.680583 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:21.646037  2779 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:21.574160  2825 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.484s
W20260709 12:25:21.654044  2796 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.004s	sys 0.012s
W20260709 12:25:21.690913 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 130507us
I20260709 12:25:21.677750 30990 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:21.682008 31136 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:21.695111 30984 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.695678 30986 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:21.697849  2584 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:21.698833  2822 consensus_meta.cc:287] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.890s	user 0.007s	sys 0.007s
I20260709 12:25:21.699764 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:21.701347 31664 connection.cc:256] RPC call timeout handler was delayed by -1.7433677229996647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
W20260709 12:25:21.701792 31664 connection.cc:256] RPC call timeout handler was delayed by -1.6971045139998751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
W20260709 12:25:21.702098 31664 connection.cc:256] RPC call timeout handler was delayed by -1.5798221559998638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 12:25:21.716420 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.742291 30987 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:21.703423  2779 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:21.767477 31664 connection.cc:256] RPC call timeout handler was delayed by -1.5236313810000865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
W20260709 12:25:21.771603 31664 connection.cc:256] RPC call timeout handler was delayed by -1.3703142730000764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27080000000000004s for a response
W20260709 12:25:21.772080 31664 connection.cc:256] RPC call timeout handler was delayed by -1.2916014979996362s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 12:25:21.730254  1995 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:21.730751  2702 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:21.731042  2584 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:21.703208  2784 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:25:21.801649 31059 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:21.805325 31059 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:25:21.806975 31059 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:21.770573  2846 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:21.796550  1995 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.753708  2798 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:21.829541 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:21.832268 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:21.838469 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 222097us
I20260709 12:25:21.839993 30886 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:21.809667  2702 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:21.843192 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.846434 31058 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:21.851960 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:21.848620 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:21.853389  2807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.016s	sys 0.019s
W20260709 12:25:21.853971 31065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.017s	sys 0.026s
W20260709 12:25:21.854508 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.820102  2723 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:21.836112  1995 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:21.856539 31065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.017s	sys 0.026s
I20260709 12:25:21.845525  2702 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.875545 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.876835 30987 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:21.877820 31058 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:21.890435  2798 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:21.892349  2816 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.069s	user 0.013s	sys 0.000s
I20260709 12:25:21.872267  2702 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.906561 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:21.926757 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:21.927498 30985 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
W20260709 12:25:21.930480 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 81933.4us
I20260709 12:25:21.938851 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:21.942066 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:21.891530  2723 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:21.954016 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:21.940768  2851 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.755s	user 0.004s	sys 0.000s
I20260709 12:25:21.938015  2599 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.970232 30985 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:21.966756 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:21.930248  2798 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:25:21.929792  2816 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:25:21.945675  2867 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:21.974066 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:21.986312 30992 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:21.986805 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:21.910329  2584 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:21.988305 30992 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:21.987604 30984 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:21.905158  2779 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:21.957504  2874 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.931116  2875 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:25:21.972565  2241 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:21.994477 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 84911.9us
I20260709 12:25:22.003046 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:22.004725 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:22.012486 30983 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:25:22.017372 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.048241 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:21.973338  2599 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:21.997478  2869 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:21.997689  2871 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:22.103212 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 447961us
I20260709 12:25:22.048856 30992 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:22.048158  2872 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:22.135215  2870 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
W20260709 12:25:22.163232  2873 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
W20260709 12:25:22.166908  2864 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
W20260709 12:25:22.168072  2858 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.462s	user 0.001s	sys 0.000s
I20260709 12:25:22.104233  2869 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.043542  2396 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:25:22.172734 30992 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.103699  2871 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.122506  2726 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:22.006409  2874 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:22.190665 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.237018 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 240314us
I20260709 12:25:22.249975 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1004) took 1248 ms. Trace:
I20260709 12:25:22.250463 31065 rpcz_store.cc:276] 0709 12:25:21.001643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.010198 (+  8555us) service_pool.cc:224] Handling call
0709 12:25:21.024233 (+ 14035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.031585 (+  7352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.031627 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.112994 (+ 81367us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b2c00000a78. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:25:21.113393 (+   399us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.862333 (+748940us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:21.862374 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.862458 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.237818 (+375360us) raft_consensus.cc:1685] finished
0709 12:25:22.237847 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.249893 (+ 12046us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6730624,"thread_start_us":748527,"threads_started":1,"wal-append.queue_time_us":827637}
W20260709 12:25:22.256206 30990 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.000s
I20260709 12:25:22.257598 30990 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
W20260709 12:25:22.275650 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.173473  2869 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.181749  2726 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.128775  2807 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 12:25:22.244326  2868 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:22.181423  2871 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:22.267972  2858 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 12:25:22.348126  2879 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:25:22.358889  2866 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.437s	user 0.001s	sys 0.000s
I20260709 12:25:22.358520  1995 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:22.359738  2861 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.434s	user 0.001s	sys 0.000s
W20260709 12:25:22.361670  2877 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 12:25:22.368741  2702 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:22.371222  2599 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:22.371543  2702 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:22.373612 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 269586us
W20260709 12:25:22.374601 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 136524us
I20260709 12:25:22.376309 30987 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:22.372817 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1005) took 1370 ms. Trace:
I20260709 12:25:22.377177 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:22.374099 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.388258 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:22.391773 30992 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:25:22.392125 30992 pending_rounds.cc:85] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 8
I20260709 12:25:22.392339 30992 pending_rounds.cc:107] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:25:22.392581 30992 raft_consensus.cc:2889] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:22.394518 31139 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.000s	sys 0.020s
W20260709 12:25:22.395295 31139 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.000s	sys 0.021s
W20260709 12:25:22.378104 31108 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.002s	sys 0.015s
W20260709 12:25:22.408170 31108 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.002s	sys 0.015s
I20260709 12:25:22.409288 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 678) took 1288 ms. Trace:
I20260709 12:25:22.409686 31140 rpcz_store.cc:276] 0709 12:25:21.120886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.121033 (+   147us) service_pool.cc:224] Handling call
0709 12:25:21.130716 (+  9683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:21.477415 (+346699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.387819 (+910404us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b540012cfd0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:22.409231 (+ 21412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10526592}
I20260709 12:25:22.412147 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:22.297024  2726 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:22.416270 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:22.417410 31136 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:22.415611 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:22.418510 30992 raft_consensus.cc:2376] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13. Re-sending same reply.
I20260709 12:25:22.418637 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:22.419446 31065 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:22.427213 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:22.376911 31062 rpcz_store.cc:276] 0709 12:25:21.001939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.010136 (+  8197us) service_pool.cc:224] Handling call
0709 12:25:21.032164 (+ 22028us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.054885 (+ 22721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.054926 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.079593 (+ 24667us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.561034 (+481441us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:21.561075 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.561136 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.372623 (+811487us) raft_consensus.cc:1685] finished
0709 12:25:22.372642 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.372761 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":636,"raft.queue_time_us":17259,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":15247,"thread_start_us":424484,"threads_started":1,"wal-append.queue_time_us":596666}
I20260709 12:25:22.164968  2872 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:22.371984  2723 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:22.437579 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:22.438454 31060 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:22.377291  2816 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:22.376693  2868 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:22.433224 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:22.387120  1995 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:22.421178  2729 consensus_meta.cc:287] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.801s	user 0.009s	sys 0.004s
I20260709 12:25:22.428331  2867 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:22.466755 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:22.467864 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:22.427456  2241 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:22.476675 31140 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:22.476949 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:22.477594 31060 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:25:22.445473  2742 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.392s	user 0.012s	sys 0.001s
I20260709 12:25:22.479321 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:22.482873 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:22.484054 31136 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:22.490310 31143 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:22.500458 31142 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:22.489924  2874 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:22.478578  1995 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:22.508296 31144 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:22.539067 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.508880 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:22.469954  2723 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:22.509982  2241 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:22.543793 31143 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:22.463860  2729 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:22.512221 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:22.514772 31033 connection.cc:256] RPC call timeout handler was delayed by -1.1297086300000956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24820000000000003s for a response
I20260709 12:25:22.520283  2584 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:22.574747  2584 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.575317  2584 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.475437  2878 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:22.460757  2816 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:22.580662  2816 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:22.539994 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:22.510615 31138 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:22.481128  2702 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 12:25:22.593364 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 81210.7us
I20260709 12:25:22.483758  2742 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:22.618306 31140 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:25:22.689098 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.694149 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:22.694942 31137 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.697023 31063 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:22.714615 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 94554.8us
I20260709 12:25:22.743431 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:22.747095 30986 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:22.617611  2807 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:22.844822 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:22.871682 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:22.713833 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:22.761255 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 171673us
I20260709 12:25:22.826493 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:22.908918 31142 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:22.874817 30987 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:22.784885 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:22.865826 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:22.888542 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.783632 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:22.769532 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 286717us
I20260709 12:25:22.876695 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:22.912436 30990 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:22.753244 31664 leader_election.cc:336] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.509s (SENT)
W20260709 12:25:22.909237 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.657s (SENT) [suppressed 60 similar messages]
W20260709 12:25:22.944217 31033 leader_election.cc:336] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.749s (SENT)
I20260709 12:25:22.944999 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:22.839896  2859 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.200s	user 0.015s	sys 0.013s
I20260709 12:25:22.948167 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2458) took 1866 ms. Trace:
I20260709 12:25:22.948606 31061 rpcz_store.cc:276] 0709 12:25:21.081768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.083075 (+  1307us) service_pool.cc:224] Handling call
0709 12:25:21.098530 (+ 15455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.098943 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.098983 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.099514 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.147418 (+ 47904us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:21.147456 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.147524 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.543543 (+1396019us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:22.947821 (+404278us) raft_consensus.cc:1685] finished
0709 12:25:22.947848 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.948096 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1458,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":21378,"spinlock_wait_cycles":9001728,"thread_start_us":47527,"threads_started":1,"wal-append.queue_time_us":539887}
I20260709 12:25:22.750706  2875 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.871006  2742 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:22.872004  2824 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:22.907238  2876 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:22.907291  2807 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.779167  2874 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.196s
W20260709 12:25:22.963614  2877 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:25:22.769034  2888 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:22.975888 31144 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:22.920691  2900 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:22.977610 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 224388us
I20260709 12:25:22.695840  2887 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.351s
W20260709 12:25:22.984390 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 53468.6us
I20260709 12:25:22.970605  2824 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:22.769291  2906 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.228s
I20260709 12:25:22.712776  2867 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:22.978312  2872 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:22.699638  2868 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:22.992444  2780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.542s	user 0.011s	sys 0.012s
W20260709 12:25:23.002668  2899 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
I20260709 12:25:23.079032 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:23.062691 31109 connection.cc:256] RPC call timeout handler was delayed by -1.345818711999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
W20260709 12:25:23.083127 31109 connection.cc:256] RPC call timeout handler was delayed by -1.146018711999659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 12:25:23.083473 31109 connection.cc:256] RPC call timeout handler was delayed by -1.1099991589999263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.48350000000000004s for a response
W20260709 12:25:23.083766 31109 connection.cc:256] RPC call timeout handler was delayed by -1.1010187119995862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 12:25:23.084059 31109 connection.cc:256] RPC call timeout handler was delayed by -1.0731187119999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
W20260709 12:25:23.084326 31109 connection.cc:256] RPC call timeout handler was delayed by -1.0479187119999551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 12:25:22.984923  2807 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:22.970634  2875 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:23.044618  2901 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.351s	user 0.001s	sys 0.001s
I20260709 12:25:23.003309  2729 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:23.082707  2780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.792s	user 0.012s	sys 0.012s
W20260709 12:25:23.064234 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 113090us
W20260709 12:25:23.066294  2792 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.606s	user 0.022s	sys 0.021s
W20260709 12:25:23.088122 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.064132  2872 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:23.100062 31139 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.000s	sys 0.007s
I20260709 12:25:23.080143 31067 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
W20260709 12:25:23.105700 31139 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.000s	sys 0.007s
I20260709 12:25:23.006516  2888 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:23.100522  2792 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.993s	user 0.023s	sys 0.021s
I20260709 12:25:23.111037  2872 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.115548 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:23.135876 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.137315 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:23.140698  2910 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:25:23.100525  2824 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.154472  2884 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
W20260709 12:25:23.161440  2896 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:25:23.162925 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 684) took 1468 ms. Trace:
I20260709 12:25:23.163316 31135 rpcz_store.cc:276] 0709 12:25:21.694577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.694728 (+   151us) service_pool.cc:224] Handling call
0709 12:25:21.698540 (+  3812us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.851806 (+153266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.851838 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.877222 (+ 25384us) log.cc:844] Serialized 26 byte log entry
0709 12:25:22.377865 (+500643us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:22.377901 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.377961 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.162685 (+784724us) raft_consensus.cc:1685] finished
0709 12:25:23.162709 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.162855 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32410,"raft.queue_time_us":100097,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":2974,"thread_start_us":533570,"threads_started":2,"wal-append.queue_time_us":1082768}
I20260709 12:25:23.165688 31137 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:25:23.167042 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 66133.5us
I20260709 12:25:23.168169 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:23.168969 31137 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:23.170212 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:23.173843  2918 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:25:23.174695  2928 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 12:25:23.175189  2891 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:25:23.179134  2824 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:23.184482 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.148923  2792 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.192683 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:23.202112  2909 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:25:23.202759 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:23.206606 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:23.206964 30983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 702) took 1526 ms (client timeout 1768 ms). Trace:
W20260709 12:25:23.207355 30983 rpcz_store.cc:269] 0709 12:25:21.680150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.680345 (+   195us) service_pool.cc:224] Handling call
0709 12:25:23.206765 (+1526420us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400023fd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:23.206864 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558615296}
I20260709 12:25:23.208958 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.209950 30984 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:23.210350 30984 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:23.211148 30984 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.212527 30984 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:23.215358 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:23.216621 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:23.216897  2914 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:23.218176 31698 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:23.218075 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:23.212869  2742 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.234288 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:23.234905 31689 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:25:23.234957 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:23.235139  2924 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.235736 31135 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:23.236052 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:23.236652 31691 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 6.
I20260709 12:25:23.235739  2924 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.237664  2822 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:23.159636  2906 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.238088  2822 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:23.239152  2934 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.239353 30990 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:23.239823 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.239656  2822 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.240628 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 718) took 1233 ms. Trace:
I20260709 12:25:23.241150  2822 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:23.243372  2826 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:23.243762  2826 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:23.244706 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.244518  2826 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.245541  2826 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:23.246371 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.247920 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.249471 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.250762 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.251632 30985 raft_consensus.cc:2410] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:23.252703 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:23.253974 31661 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:23.257893 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.240978 30991 rpcz_store.cc:276] 0709 12:25:22.006776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.006892 (+   116us) service_pool.cc:224] Handling call
0709 12:25:23.239686 (+1232794us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:25:23.240573 (+   887us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7436160}
I20260709 12:25:23.262820 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:23.190845  2792 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:23.279521 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 129459us
I20260709 12:25:23.282681  2924 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:23.255455  2869 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.868s	user 0.008s	sys 0.003s
I20260709 12:25:23.283916  2792 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.287664 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.290409  2649 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:23.291513 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.292786 30984 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:25:23.295121  2436 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.005s	sys 0.032s
W20260709 12:25:23.295951 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.296953 30983 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:23.284786  2396 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:23.299422 31135 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:23.299896 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.297319 31060 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:25:23.301908 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.302233 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.338553 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.339502 31136 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.342779 31140 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:23.349776 30992 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:23.303829 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 720) took 1200 ms. Trace:
W20260709 12:25:23.334776 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 47215.7us
W20260709 12:25:23.338973 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.334785 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.340729 31144 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:23.258967  2928 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.340060 31137 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:23.280841  2696 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:23.351254 30984 rpcz_store.cc:276] 0709 12:25:22.099784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.102505 (+  2721us) service_pool.cc:224] Handling call
0709 12:25:23.209612 (+1107107us) spinlock_profiling.cc:243] Waited 898 us on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:25:23.209821 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.300346 (+ 90525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.300500 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065664}
I20260709 12:25:23.353480 30988 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:25:23.301375  2436 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.005s	sys 0.032s
W20260709 12:25:23.301728  2584 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.726s	user 0.014s	sys 0.000s
W20260709 12:25:23.286458  2732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.005s	sys 0.005s
W20260709 12:25:23.359015 31141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.002s	sys 0.009s
W20260709 12:25:23.370210 31141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.002s	sys 0.010s
W20260709 12:25:23.358319 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.362509 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.363962 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.365048 31138 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:25:23.362538 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:23.379917 30986 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:23.361145 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.239693  2934 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.377256 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.356366  2928 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:23.367187  2871 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.954s	user 0.013s	sys 0.000s
I20260709 12:25:23.378382 30990 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:23.382498  2871 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.382990 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.383484 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.383584 30990 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:23.384122 30986 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:23.384428 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.382021  2928 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.384938 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.386577 31693 raft_consensus.cc:1217] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Deduplicated request from leader. Original: 8.8->[10.9-10.9]   Dedup: 10.9->[]
I20260709 12:25:23.387059 31693 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:23.387117  2934 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:23.391790  2878 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:23.386618 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.393889 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:23.395915  2913 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.241s	user 0.003s	sys 0.000s
W20260709 12:25:23.397125  2805 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:25:23.396937 30992 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:23.399750 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.400274 30987 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.400648  2872 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:23.359812  2696 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.385039 30990 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:23.385602 30989 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:23.392464  2878 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:23.384547  2871 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:23.371966  2732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.005s	sys 0.019s
I20260709 12:25:23.402760 30986 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:23.381253 30988 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:23.403957 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:23.384887  2726 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.950s	user 0.002s	sys 0.006s
I20260709 12:25:23.425071 30992 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 5.
I20260709 12:25:23.378549 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.425486 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:23.370559 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 67464.4us
I20260709 12:25:23.426694 30988 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:23.380777 30983 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:23.414278 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.423345  2917 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:23.427671 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.428045 30990 raft_consensus.cc:2376] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14. Re-sending same reply.
I20260709 12:25:23.428478 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.430725 30983 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:23.426084 30991 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:23.431407 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.430805 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.432520 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.301438  2869 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.434405  2875 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:23.437999  2940 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:23.438587  2940 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.440807  2940 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:23.442003  2888 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:23.447363 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.447889 30983 raft_consensus.cc:2376] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8. Re-sending same reply.
I20260709 12:25:23.412974  2696 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.452966  2939 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.447206  2726 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.457688 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:23.458268 31060 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.460314 30984 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:23.462756  2726 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:23.466073  2732 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:23.466527  2732 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:23.466892  2732 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.468925  2792 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.375037  2584 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.474958  2807 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.475991  2584 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:23.477072  2807 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:23.477825 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:23.478338 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 115989us
I20260709 12:25:23.480476 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.481479  2881 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:23.481557 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.486639 30987 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:25:23.487555 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.451617  2852 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:25:23.496973 30988 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:25:23.489133  2863 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.734s	user 0.008s	sys 0.012s
I20260709 12:25:23.490276 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.491025 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.498009 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.500066 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:23.516155 31067 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:23.501782 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:23.517623 31058 raft_consensus.cc:2368] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:23.518335 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:23.498567 31060 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:23.429766  2917 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:23.527179 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 155432us
I20260709 12:25:23.528776 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:23.530465 31067 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:23.531927 31067 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:23.528368 31058 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:23.525663 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:23.526127 31033 leader_election.cc:336] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.577s (SENT)
I20260709 12:25:23.535528 30955 leader_election.cc:400] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:23.537379 30955 leader_election.cc:403] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:23.538439 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:23.539000 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 57524.8us
I20260709 12:25:23.515127 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:23.548332 31065 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:23.549754 31065 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.499225 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.551204 30984 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:23.552214 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.552889 30984 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:23.505074 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:23.555002 31138 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:23.509919 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.557015 30983 raft_consensus.cc:2410] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:25:23.539186 31695 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:25:23.557936 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.540490 31058 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:23.541608 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:23.514788  2878 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.562444 31068 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:23.505992 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:23.563444 31137 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.553864 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.564949 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.565562 30984 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:23.566203 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.567106 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.567678 30984 raft_consensus.cc:2368] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 7. Current term is 8.
I20260709 12:25:23.568322 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.570077  2878 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:23.572810 31065 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:23.573065 31137 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:23.581579 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:23.585173 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:23.586189 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:23.588928 31144 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:23.583400 31140 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:23.501969  2914 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:23.592483 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50954.7us
I20260709 12:25:23.648284 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.643477  2890 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:23.649943  2584 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.650035 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.653057 31142 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:23.651787 31137 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:23.659687  2914 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:23.662595  2934 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.663112  2934 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.664445  2890 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:23.664856  2890 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:23.665270  2934 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:23.670621 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 144518us
I20260709 12:25:23.671299 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.671734  2584 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:23.672214  2914 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:23.644430 31691 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.676378  2881 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.682389 30985 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:23.681794 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:23.684810  2796 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:23.685346  2796 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:23.686601  2881 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.687125  2881 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:23.687572  2881 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.690400  2937 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.691047  2937 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.507678  2875 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.187s
I20260709 12:25:23.694801  2934 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:23.592468  2805 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:25:23.695438  2934 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:23.696157  2805 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:23.696633  2805 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:23.696455  2934 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.696895  2890 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.584228  2824 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.698714 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:23.700798  2914 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.700912  2934 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.701241  2914 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:23.701654  2914 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.583972  2940 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.702756  2940 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.705230  2940 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:23.705798 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 278594us
I20260709 12:25:23.655356 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:23.714666 31135 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.712029 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.654992 31689 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:23.672588 31033 leader_election.cc:336] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.076s (SENT)
I20260709 12:25:23.724959 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:23.731581 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:23.723984 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:23.735114 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.736649 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:23.736720 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 691) took 1228 ms. Trace:
I20260709 12:25:23.735446 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.694056  2937 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:23.737186 31141 rpcz_store.cc:276] 0709 12:25:22.508080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.509309 (+  1229us) service_pool.cc:224] Handling call
0709 12:25:22.509647 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:22.543867 (+ 34220us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:22.621930 (+ 78063us) log.cc:844] Serialized 14 byte log entry
0709 12:25:23.429960 (+808030us) log.cc:844] Serialized 14 byte log entry
0709 12:25:23.500784 (+ 70824us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b180044c380. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:23.524652 (+ 23868us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.586692 (+ 62040us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.586876 (+   184us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:23.586925 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.587022 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.736522 (+149500us) raft_consensus.cc:1685] finished
0709 12:25:23.736544 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.736663 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":191939,"prepare.queue_time_us":44765,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":364944,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":17551,"spinlock_wait_cycles":44712064,"thread_start_us":583970,"threads_started":1,"wal-append.queue_time_us":727865}
I20260709 12:25:23.539157  2942 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.744149 31062 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:23.709559  2914 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.741606 31136 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.579527  2869 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:23.746289 31136 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:23.752681 30983 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:23.754384 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:23.757071 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:23.759002 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:23.759684 31141 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:23.736794  2905 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.026s	sys 0.046s
W20260709 12:25:23.762673 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 90130.2us
W20260709 12:25:23.762928  2905 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.029s	sys 0.046s
W20260709 12:25:23.763381 31033 leader_election.cc:336] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.162s (SENT)
I20260709 12:25:23.763819 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.765849  2893 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.766953 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.768767 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:23.769227  2911 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.770015  2911 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:23.770393  2911 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.770781  2911 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:23.770469  2930 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:25:23.771857  2930 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:23.772312  2930 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:23.775547  2875 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:23.776513  2875 consensus_queue.cc:1243] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:23.736738  2939 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:23.780629 31144 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:23.773315  2930 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.756254  2867 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.785494 31141 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 12:25:23.744531  2942 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:23.786911 31067 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:23.697723  2805 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.776969  2822 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:23.799212 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.736672  2779 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:25:23.739836  2742 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:25:23.799173 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.748039  2826 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:23.788723 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 61386.4us
I20260709 12:25:23.744422  2740 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:25:23.803640 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:23.803936 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.804289 31138 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:23.781173  2396 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:23.844869 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.839991  2945 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:25:23.757654  2937 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:25:23.755352  2696 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:23.843748 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56304.6us
I20260709 12:25:23.727525  2796 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:23.800938  2942 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:23.845299 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.841884  2939 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:23.806351 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:23.836225  2930 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.801787  2924 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:23.803057  2822 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:23.804278  2792 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:23.805955  2941 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.902897 31693 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:23.846966  2805 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:23.895454 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 96344.7us
I20260709 12:25:23.786914  2893 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:25:23.786002  2940 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:25:23.801576  2875 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
W20260709 12:25:23.910465 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 67285.7us
I20260709 12:25:23.911378 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:23.924603 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.794329  2917 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.913936  2796 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.925628  2917 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.928802  2396 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.846452  2826 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.845942  2928 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.922052 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:23.846573  2881 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:23.896560  2903 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.957s	user 0.000s	sys 0.005s
I20260709 12:25:23.918823  2924 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.935801  2867 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.896692  2868 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:25:23.896814  2915 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 12:25:23.941845  2928 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:23.969585 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.938730  2897 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:23.923722  2941 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:23.955132  2934 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:23.954294  2867 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:23.986711 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75441.1us
I20260709 12:25:23.987455 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:23.987870 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 63349.2us
I20260709 12:25:23.988515 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:23.993410 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:23.981700  2784 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.984441  2875 consensus_queue.cc:1243] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:23.984081  2897 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:23.985181  2867 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.993428  2869 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.986991  2696 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:23.992789  2914 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:23.963531  2822 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.002882  2878 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:24.012099  2875 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:24.011844  2924 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.008657  2928 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:24.029394 31135 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:24.018213  2869 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:24.020960  2824 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:23.996752  2934 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.020851  2878 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:24.004685  2784 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.022056  2826 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.027688  2875 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:24.032210  2822 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.033643  2869 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.040513 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 682) took 2408 ms. Trace:
I20260709 12:25:24.044914 30990 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:24.035880  2889 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:24.047931 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.050683 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2471) took 1479 ms. Trace:
I20260709 12:25:24.035789  2878 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:24.061666 31064 rpcz_store.cc:276] 0709 12:25:22.571313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.571571 (+   258us) service_pool.cc:224] Handling call
0709 12:25:22.571865 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.572070 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:22.833527 (+261457us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.936042 (+102515us) log.cc:844] Serialized 14 byte log entry
0709 12:25:24.050234 (+1114192us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b180034a980. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287
0709 12:25:24.050337 (+   103us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:24.050388 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:24.050430 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.050497 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.050627 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50971,"prepare.queue_time_us":226012,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":8904,"raft.queue_time_us":211988,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":14872,"spinlock_wait_cycles":22808576,"thread_start_us":250331,"threads_started":3}
I20260709 12:25:24.052925  2875 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:24.038926  2732 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
W20260709 12:25:24.039774 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51068.4us
I20260709 12:25:24.059409 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:24.051517  2852 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:24.067409 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:24.068192 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:24.081974 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:24.068769 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:24.073593  2941 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:24.060210 31139 rpcz_store.cc:276] 0709 12:25:21.631914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.632091 (+   177us) service_pool.cc:224] Handling call
0709 12:25:21.660092 (+ 28001us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.728617 (+ 68525us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:22.395816 (+667199us) log.cc:844] Serialized 14 byte log entry
0709 12:25:23.106450 (+710634us) log.cc:844] Serialized 14 byte log entry
0709 12:25:23.167944 (+ 61494us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.194264 (+ 26320us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.194403 (+   139us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:23.221858 (+ 27455us) log.cc:844] Serialized 14 byte log entry
0709 12:25:23.222107 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.475710 (+253603us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b180005ecc0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:23.483494 (+  7784us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b5400062a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:23.483560 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.492610 (+  9050us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:23.766801 (+274191us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b180005ecc0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:23.929105 (+162304us) spinlock_profiling.cc:243] Waited 73.7 ms on lock 0x7b180005ecc0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:24.040273 (+111168us) raft_consensus.cc:1685] finished
0709 12:25:24.040298 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.040435 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91774,"prepare.queue_time_us":399141,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":15909,"raft.queue_time_us":313340,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":12252,"spinlock_wait_cycles":356015872,"thread_start_us":1302875,"threads_started":3,"wal-append.queue_time_us":222850}
I20260709 12:25:24.082182 30992 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.071434  2784 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:24.083010 30986 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.075909  2826 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:24.090083 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:24.091099  2875 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:24.056980  2881 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.096681 30992 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
W20260709 12:25:24.106348 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 47003.1us
I20260709 12:25:24.107399 31033 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:24.108420 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:24.109375  2950 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.251s	user 0.000s	sys 0.003s
I20260709 12:25:24.110504  2940 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:24.110967  2940 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.111402  2940 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:24.113519 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2163) took 2484 ms. Trace:
I20260709 12:25:24.113813 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:24.113873 31063 rpcz_store.cc:276] 0709 12:25:21.629024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.700813 (+ 71789us) service_pool.cc:224] Handling call
0709 12:25:21.701108 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.717612 (+1016504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.113299 (+1395687us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54000c6f54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:24.113462 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2123473664}
I20260709 12:25:24.093451  2939 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:25:24.116232  2868 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:24.116622  2868 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:24.116848  2934 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:24.116990  2868 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.128257 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:24.103838  2941 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:24.124572  2784 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:24.086701  2732 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.130460  2934 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:25:24.151435 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:24.156023 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:24.158334 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:24.159777 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:24.140192  2952 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:25:24.163722 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:24.126150  2824 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:24.171464 30986 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:24.169901 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:24.172426 31141 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.172936 31138 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:24.173391 31135 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:24.164347  2784 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:24.192530 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 82699.5us
I20260709 12:25:24.173854 31143 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.209733 31141 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:24.126132  2826 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.130321  2930 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:25:24.141052  2860 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:25:24.144621  2924 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:25:24.143087  2599 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:25:24.142575  2928 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:25:24.243813 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:24.216661  2784 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:24.078732  2897 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.223099  2893 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:24.169391  2954 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:25:24.256105 31135 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:24.270625 31143 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
W20260709 12:25:24.272253 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 108571us
I20260709 12:25:24.268621  2942 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.277020  2826 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.277254  2942 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:24.269961  2893 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.283993  2893 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.284468  2893 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:24.296914  2959 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:25:24.312165 31138 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:24.284961  2893 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:24.331722 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 53313.3us
I20260709 12:25:24.305972  2940 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.354173 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:24.356215 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.326321  2826 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:24.366747 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:24.367914 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:24.329046  2942 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:24.369009 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:24.372677 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:24.331475  2906 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:24.384653 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.372991  2942 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:24.395596 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.383863  2867 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.391685  2972 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:24.418569 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.418989  2979 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.419103  2944 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.736s	user 0.003s	sys 0.000s
I20260709 12:25:24.383241  2906 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.397862  2732 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:24.294080 30877 maintenance_manager.cc:643] P 0423d91b75f1428faccc573fcbf419a1: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 9.875s	user 0.206s	sys 0.093s Metrics: {"bytes_written":30095,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":978325,"dirs.run_cpu_time_us":413,"dirs.run_wall_time_us":19571,"drs_written":1,"lbm_read_time_us":41576,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":1,"lbm_write_time_us":303722,"lbm_writes_1-10_ms":5,"lbm_writes_10-100_ms":7,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":21,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":158233,"threads_started":1,"update_count":19}
I20260709 12:25:24.443639 31696 raft_consensus.cc:2368] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11.
I20260709 12:25:24.452930 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:24.453770 31689 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:24.430730  2824 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:24.443781  2934 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:24.450901  2954 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.432085  2973 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:24.457312  2822 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:24.467833 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 95927.7us
I20260709 12:25:24.431195  2875 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:25:24.411298  2869 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:24.471134 31109 leader_election.cc:336] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.540s (SENT)
I20260709 12:25:24.472214 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:24.473742 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.477909 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:24.464588  2867 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:24.475853 31696 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:24.481935 31691 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:24.482613 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:24.483608 31142 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:24.486393  2732 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:24.488914  2889 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:24.474432  2956 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:24.489809 31693 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:24.491508 31689 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:24.494825 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.497771 30878 maintenance_manager.cc:419] P 0423d91b75f1428faccc573fcbf419a1: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10109 bytes on disk
I20260709 12:25:24.467949  2954 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.510049  2890 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.544865  2954 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.545259  2792 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:24.545624  2792 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.546407  2792 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.546952 30877 maintenance_manager.cc:643] P 0423d91b75f1428faccc573fcbf419a1: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":313,"lbm_reads_lt_1ms":4}
I20260709 12:25:24.453087  2979 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.549309  2979 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.478837  2868 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.449321  2942 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.502700  2867 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:24.474153  2986 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:24.480118  2893 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.553794  2986 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.556233  2986 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.508437 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:24.525548  2784 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.557171 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:24.557374 31691 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.508040 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:24.558686 31689 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:24.562328  2867 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.563382 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:24.564669 31140 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:25:24.567070 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:24.567282 31068 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:24.567873 31139 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:24.513826 31061 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:24.510586 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.525434 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:24.503671  2911 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.003s
W20260709 12:25:24.498133  2796 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.000s
I20260709 12:25:24.519629  2906 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:24.547552  2792 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.509435 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:24.591030 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:24.552462 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:24.552939  2953 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:24.553078  2868 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.517163  2869 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:24.564725  2867 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.511209 31137 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:24.568641 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:24.619865 31060 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:24.620002 31142 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.513288 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:24.581379  2931 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:24.581790  2890 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:24.499807  2940 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.493971  2878 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.582562 31064 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:24.583003 31695 raft_consensus.cc:2368] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:24.583194 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:24.634279 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 51150.7us
I20260709 12:25:24.635123 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:24.636077  2649 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:24.636155 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:24.582038  2924 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:24.585744 31691 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:24.586177 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:24.637758 31058 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.619022 31068 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:24.622025 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:24.630498 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 78107us
I20260709 12:25:24.638849 31063 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.558686  2784 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:24.639293 31109 leader_election.cc:400] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11.
I20260709 12:25:24.639734 31109 leader_election.cc:403] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:24.639206 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:24.631861 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:24.636849  2649 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.641747 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:24.594897  2953 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:24.636992  2889 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:24.649870 31065 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:24.632117  2906 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:25:24.585400  2911 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.638809  2924 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:24.644352  2740 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11.
I20260709 12:25:24.644546  2917 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:24.646319  2860 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:24.655632  2928 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:24.679001 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:24.689661 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:24.657899  2889 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.641639  2732 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:25:24.651861  2890 raft_consensus.cc:2764] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:25:24.693120 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.675663 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:24.699199 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:24.679104  2436 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:24.703751  2436 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.704610  2436 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.705819  2436 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:24.675385  2784 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:25:24.701891 31058 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:24.706820 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:24.708644 31061 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:24.674641  2696 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:25:24.712467  2894 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:24.713254 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:24.717574 31063 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:24.674860  2986 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:25:24.692358  2860 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.694166  2928 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.710685  2869 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:24.763988 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 124833us
W20260709 12:25:24.767220 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59904.3us
W20260709 12:25:24.767416 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 118622us
I20260709 12:25:24.768175 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:24.710201  2931 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:25:24.734053  2649 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.777527  2983 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:24.779109 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:24.761524  2867 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:25:24.780015 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:24.700197  2953 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.781490 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:24.781713  2953 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.781899 31061 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:24.782450  2892 raft_consensus.cc:2764] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:24.783185  2892 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:24.783618  2892 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:24.785609  2702 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:24.709141  2924 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.786121  2702 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:24.789613 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:24.786659  2702 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:24.801474  2702 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.803622  2702 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.803761  2860 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.806922  2897 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:24.807360  2911 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.807327  2897 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:24.808100  2889 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:24.777544  2988 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:25:24.810061  2897 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.774392  2869 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:24.810158 31061 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:24.824317 31142 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:24.830660  2999 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:24.784469  2892 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.811653  2897 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:24.821528  2784 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:24.824844  2702 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:24.816361  2878 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:24.850697 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 210038us
I20260709 12:25:24.854682 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:24.840328  2924 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:24.816861  2979 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:25:24.839869  2999 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:24.810468  2928 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.855796  2784 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:24.815311  2869 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.850241  2599 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:25:24.840087  2989 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:25:24.859452  2999 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.888000 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:24.898375  3000 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:24.892508  2737 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:24.899461  2737 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.900362  2737 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.901578  2737 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:24.903282  2906 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:24.903693  2906 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:24.904497  2906 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.921283  2826 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:24.921747  2826 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:24.922591  2826 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.923775  2897 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.923832  2826 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.924609  2897 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:24.927112  2893 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:24.927496  2893 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:24.928552 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:24.930130 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 107591us
I20260709 12:25:24.931109 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:24.934847 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:24.943354 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.944182 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:24.953205 31142 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:24.932343  2740 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.963392 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.964640 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:24.969746 31033 leader_election.cc:336] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.713s (SENT)
I20260709 12:25:24.972675 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:24.937201  2935 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:24.973783 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:24.974767 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:24.978588 31142 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:24.963006 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:24.976282 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:24.976368 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:24.981462 31143 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:25:24.981812 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 209267us
I20260709 12:25:24.981884 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.938601  2986 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:24.987754 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:24.961822  2979 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:24.991120 31033 leader_election.cc:336] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.986s (SENT)
I20260709 12:25:24.996598 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.992666 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:24.997406 31698 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:24.996770 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:25.002401 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:24.992048  2702 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:25.001835 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:25.009581 31140 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:25.010829  2911 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:25.011369  2911 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:25.012277  2911 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.017978 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:24.993999  2784 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:24.959710  2893 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.022979  2893 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.009012  2935 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:25.027081  2935 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:25.029417 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:24.990061  2897 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.988572  2986 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.034936  2986 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:24.979998  2740 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:25.038462 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.045080 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:25.006038  2984 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:25.046420 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:25.047160 31694 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:25.052502  2868 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:25.052887  2868 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:25.055918 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:25.062071  2984 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:25.062337 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 586145us
I20260709 12:25:25.062661  2984 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.063084 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:25.009936  2979 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:25.064272 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:25.064144  2979 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.064826  2984 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:24.994788  2999 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.065919  2973 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:25.066854  2824 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:25.066958  2999 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:25.069478  2942 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:25.069726 30955 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:25.071058  2930 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:25.071429  2930 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:25.072472  2930 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.073434  2930 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:25.053628  2868 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.073982 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:25.074913  2868 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:25.075636 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:25.076195 30992 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.078835 30984 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 12:25:25.079851 30983 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:25.081074 30983 pending_rounds.cc:85] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 9
I20260709 12:25:25.081341 30983 pending_rounds.cc:107] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:25:25.081575 30983 raft_consensus.cc:2889] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:25.084913 30992 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:25.084992 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:25.085628 30989 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:25.063203  2740 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:25.087280 31691 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 6.
I20260709 12:25:25.071463  2982 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.079893 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:25.067067  2973 raft_consensus.cc:2764] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:25.090960  2986 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:25.096091 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 86005us
W20260709 12:25:25.106630 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 107865us
W20260709 12:25:25.119122 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49477.7us
I20260709 12:25:25.127245 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:25.123286 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 58014.1us
I20260709 12:25:25.129422 30987 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 8: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:25.127585 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.130733 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.127290  2982 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.135924 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:25.137951  2982 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:25.138135 31033 leader_election.cc:336] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.622s (SENT)
W20260709 12:25:25.139690 31033 leader_election.cc:336] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.657s (SENT)
I20260709 12:25:25.140219 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:25.141235 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:25.141847 30987 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 7: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:25.145288 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:25.145880 31063 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:25.149672  3001 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:25.150585  3001 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:25.150949  3001 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:25.151660  2986 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.151825  3001 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.152369  2935 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.153090  3001 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.154503 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:25.155071 31694 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:25.155066 31689 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:25.148943 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:25.156118 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:25.156925 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:25.157620 30987 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.158505 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:25.161448 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:25.163055 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:25.164259 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:25.169785 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:25.126735  2980 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:25.166857  2892 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.167016  2869 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:25.165627  2868 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:25.187194 31696 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:25.167258  2984 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:25.192987  2984 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:25.193404  2984 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:25.185985 31033 leader_election.cc:336] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.070s (SENT)
I20260709 12:25:25.188875 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:25.192334  2893 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:25.196135 31141 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.197000 30987 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
W20260709 12:25:25.196941 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 44207.1us
W20260709 12:25:25.197804 31109 leader_election.cc:336] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 2.172s (SENT)
I20260709 12:25:25.198068 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:25.198182 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:25.171104  2986 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.198832 30987 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.179936  2878 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:25.212011 31033 leader_election.cc:336] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.103s (SENT)
I20260709 12:25:25.213346 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2418) took 1908 ms. Trace:
W20260709 12:25:25.214733 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.181918  2980 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:25.213709 30990 rpcz_store.cc:276] 0709 12:25:23.304862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.446500 (+141638us) service_pool.cc:224] Handling call
0709 12:25:24.044603 (+598103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.060863 (+ 16260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.213170 (+1152307us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b540014e6d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:25.213291 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374742784}
I20260709 12:25:25.223083 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:25.223652 30990 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:25.224434 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:25.225142 30990 raft_consensus.cc:2410] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:25.227319 31141 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:25.202356  2869 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:25.210289  2732 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:25.234916 30987 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
W20260709 12:25:25.232661 31033 leader_election.cc:336] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.162s (SENT)
I20260709 12:25:25.240604 30989 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:25.242195 30990 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:25:25.222729  2980 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:25.245056 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.260690 30983 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:25:25.261075 30983 pending_rounds.cc:85] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 9
I20260709 12:25:25.261282 30983 pending_rounds.cc:107] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:25:25.261509 30983 raft_consensus.cc:2889] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:25.244393 31033 leader_election.cc:336] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.177s (SENT)
I20260709 12:25:25.247478  2869 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.264561 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:25.265183 31692 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
W20260709 12:25:25.268002 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 70205.9us
I20260709 12:25:25.270323 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:25.279302  2881 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:25.279745  2881 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:25.280632  2881 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.282119  2881 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.283834 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:25.286140  2696 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:25.286618  3001 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:25.286739  2696 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.289043  2696 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.290721 30984 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:25.286161 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:25.291991 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:25.295298 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50922.9us
I20260709 12:25:25.295763 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:25.296111 31033 leader_election.cc:336] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.226s (SENT)
W20260709 12:25:25.302208 31033 leader_election.cc:336] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.233s (SENT)
I20260709 12:25:25.302631 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:25.291661  2939 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.305265 31135 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:25.306028 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:25.306901 31033 leader_election.cc:336] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.824s (SENT)
I20260709 12:25:25.297077  2732 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.308202 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:25.309235 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:25.308010  3014 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:25:25.315641  2889 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:25.316175  2889 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:25.298095  2867 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:25.298847  3005 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:25.301584  2696 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:25.319453  3005 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.319649  2696 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:25.319897  3005 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.320130  2696 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:25.306767  2911 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:25:25.301816  3001 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.322806 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:25.307310  3015 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:25:25.324241  2996 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:25.325013 31140 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:25.325373 31139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:25.332264 31139 raft_consensus.cc:606] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:25.332718 31139 raft_consensus.cc:614] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:25.307165  2878 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.329952 31141 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 6.
I20260709 12:25:25.332569 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:25.341426 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:25.342114 31694 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 7.
I20260709 12:25:25.325925  2928 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:25:25.328547  2889 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.347543 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:25.333606  2852 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:25:25.320964  2935 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:25.331377  2996 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:25.336602  2881 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:25.358742  2649 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:25.334362  3001 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.355645 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:25.349335 31033 leader_election.cc:336] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.843s (SENT)
I20260709 12:25:25.359135  2997 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:25:25.330900  2737 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:25:25.413347 31139 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:25.421586 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:25.344969  2740 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:25.426285  2906 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:25.426434  3001 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.431448 30990 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:25.434369 30990 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:25.438356 31058 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:25.419831  2649 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:25.437075  2740 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:25.448053 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 98739.4us
I20260709 12:25:25.448789 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:25.424192  2878 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.451742 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:25.452349 31139 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:25.432775  2935 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:25.447144  2649 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:25.460659 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:25.446827  2890 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.433120  2984 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.474234  2984 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:25.475227  3028 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:25:25.471963 31139 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:25:25.477860 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.466401  2599 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:25.463050  2740 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.467403  2889 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:25.468400  3001 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:25.456715  2979 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:25.471918  2890 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.465471  2878 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.478740  2907 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:25.433522  2996 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.502981  2996 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.423923  2980 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.505116  2980 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:25.473551  2696 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.507212  2696 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.498582 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:25.500285 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:25.500460 31068 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:25.495077  3001 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.502190  2889 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:25.508771 30989 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:25.514920 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.513659  3001 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:25.521517 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:25.526759 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 394613us
I20260709 12:25:25.529958 31140 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:25.530700 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:25.544163  3031 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:25.549268  3005 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.513803  2889 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:25.552438  2889 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.555878 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:25.556133 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:25.556468 31058 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.511216  2979 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:25.572108 31139 raft_consensus.cc:2368] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
I20260709 12:25:25.580997 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:25.595981 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 65401.1us
I20260709 12:25:25.528129 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:25.528268 31692 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:25.571357  2982 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.513783  2878 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.604879 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:25.609925 31135 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
W20260709 12:25:25.610167 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 82127.4us
I20260709 12:25:25.605751 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:25.606405 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.591984 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:25.605628 31058 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:25.580608  2889 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:25.611179 31698 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:25.625255 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:25.614686 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:25.641402 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:25.641644 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:25.642390 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:25.609108  3030 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:25:25.647956 31135 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:25.573609  2937 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:25:25.643630 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:25.643956 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:25.644903 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52970.3us
I20260709 12:25:25.641430 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:25.606683  2824 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.668955 31142 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:25.619094  2942 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.674672 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55824.2us
I20260709 12:25:25.625689  2986 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:25.725939 31142 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:25.676374 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:25.680262 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:25.640861  3005 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:25.774011 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:25.774159 31136 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:25.787052 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:25.665536  2985 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.676147  3022 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:25:25.671801  2740 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.618479  2982 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:25.770392  3036 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
W20260709 12:25:25.770263  3027 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:25:25.640777  3000 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 12:25:25.596057  2935 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.791502  2937 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:25.835438  2937 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:25.836560  2937 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:25.842630  3039 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:25:25.844879  2982 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.797895  2893 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:25.848503  2893 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:25.849575  2893 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.764369  2942 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:25.816215  3037 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
W20260709 12:25:25.818231  3034 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:25:25.775286  2979 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.680891  2824 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.831773  2878 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:25.832412  2890 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:25.865348  3001 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.866406  2740 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.869861 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 268014us
I20260709 12:25:25.838037  2937 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:25.872133 31695 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:25.805179  2985 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.881362  2396 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.826121  2649 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:25.949184  2983 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:25.949757  2983 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:25.953787  2942 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.954655  3047 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.955284  3047 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.957777  3047 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:25.960697  2985 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.887992  2740 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.963150  2740 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:25.965507  3026 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.645s	user 0.005s	sys 0.000s
I20260709 12:25:25.971940  2824 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:25.973908  3001 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.974766  2930 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:25.975266  2972 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:25.975404  2930 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.976001 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:25.975813  2972 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:25.976554 31698 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:25.977766  2972 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.979226 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:25.979746 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:25.980033 30990 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:25.983565 31060 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:25.985069 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:25.985746 31060 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:25.988622 31067 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:25:25.988971 31067 pending_rounds.cc:85] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 11
I20260709 12:25:25.989190 31067 pending_rounds.cc:107] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:25:25.989418 31067 raft_consensus.cc:2889] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:25.973948 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:25.995190 31059 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:25.991488 30955 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:25.996294 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2431) took 1936 ms. Trace:
W20260709 12:25:26.000380 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 324200us
I20260709 12:25:26.004227 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.005023 30992 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:26.005251 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.006760 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:26.012603 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.012820 31060 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:25.977906  2930 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:25.996626 30988 rpcz_store.cc:276] 0709 12:25:24.059389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.060459 (+  1070us) service_pool.cc:224] Handling call
0709 12:25:24.060758 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:24.064807 (+  4049us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:25.213173 (+1148366us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b180041a520. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf
0709 12:25:25.213295 (+   122us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.590996 (+377701us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.591211 (+   215us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:25.602659 (+ 11448us) log.cc:844] Serialized 52 byte log entry
0709 12:25:25.602795 (+   136us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:25.602963 (+   168us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.773961 (+170998us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.973297 (+199336us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b180041a520. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2eef76858f 00007f2ee87a7571 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd
0709 12:25:25.996013 (+ 22716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.996079 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.996117 (+    38us) raft_consensus.cc:1685] finished
0709 12:25:25.996130 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.996242 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":838851,"prepare.queue_time_us":60116,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":3241,"raft.queue_time_us":479420,"raft.run_cpu_time_us":535,"raft.run_wall_time_us":56800,"spinlock_wait_cycles":45786240,"thread_start_us":98172,"threads_started":2,"wal-append.queue_time_us":118978}
I20260709 12:25:26.014165 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.012854 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.017426 31064 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:26.017572 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.017903 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.018302 30988 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.018515 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.019045 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.019601 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.020180 30992 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.929508  2890 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:25:25.990685  2972 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.022982  2740 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:26.013047  2875 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:26.023667  2875 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:26.023514  2740 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.024065  2875 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:26.025681  2740 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:26.006492  2954 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.012825  2737 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:25:26.026448  2954 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.028208 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.028578  2954 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.014995 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.029479 30987 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:26.015846  2969 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:26.030278 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.030812 30987 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:26.032428 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 614) took 1053 ms. Trace:
I20260709 12:25:26.032774 31691 rpcz_store.cc:276] 0709 12:25:24.987309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.987451 (+   142us) service_pool.cc:224] Handling call
0709 12:25:26.032269 (+1044818us) spinlock_profiling.cc:243] Waited 92.1 ms on lock 0x7b540062ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:26.032359 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211918208}
I20260709 12:25:26.033167  2649 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.013185  2987 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:26.015836 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.022733  2972 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.034149  2987 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:26.034731 30990 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:26.035133  2987 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.035779  2972 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.017520 31067 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:25.931937  2396 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.035517 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.039156 30990 raft_consensus.cc:2410] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:25:26.041821 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.043911 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:26.044481 31141 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:26.046262 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:26.046980 31141 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:26.049292 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:26.049852 31141 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:26.046175 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.052230 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.052776 31067 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:26.056968 30991 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:26.048231 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:26.059764 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:26.060487 31143 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:26.060721 31106 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:26.063802 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.064481 31061 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:26.063275 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:26.062114 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:26.066537 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:26.072203 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:26.072386 30992 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:26.073335 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:26.073369 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:26.073201 30987 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:26.074726 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:26.078872 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.079691 30989 raft_consensus.cc:2410] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 12:25:26.004840  2986 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:26.085258 30988 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:26.090555 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.096441 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:26.016856  3047 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:26.106462 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 57466.2us
I20260709 12:25:26.107080 31144 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:26.059512  2930 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.054600  2982 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:26.106424 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:26.074472  2972 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:26.115124 31138 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.014s	sys 0.029s
I20260709 12:25:26.123503 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.059938  2969 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:26.098552  2973 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:26.078755  2805 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:26.098753  2939 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:26.078966  2796 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:26.131911 31138 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.014s	sys 0.030s
I20260709 12:25:26.132012 30984 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:26.079131  2980 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:26.112923 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:26.107378  3014 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:26.120882  2954 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:26.106148  2970 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:25:26.133641  2972 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.110118  2737 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:25:26.141917  2939 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.124454  3047 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.127096  2860 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:25:26.129004  2930 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.138746  2792 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:25:26.159428  2396 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:26.081027  2983 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.180707 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.180804 30988 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:26.142154  2969 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.187943 31698 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:25:26.188021 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 75209.7us
I20260709 12:25:26.192075 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:26.176566  3014 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.205312 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:26.186980  2972 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:26.271266 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 66039.7us
I20260709 12:25:26.188585  2939 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:26.298712 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.322306 30988 raft_consensus.cc:2376] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13. Re-sending same reply.
I20260709 12:25:26.187155  2954 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:26.300485 30958 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:26.191303  2796 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:26.324306 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:26.329761 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.326447  2826 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:26.331025  2826 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.331372  2826 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:26.331535  2940 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:26.331965  2940 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.332297  2940 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:26.306532  3009 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:25:26.338382 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.298561  3014 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:26.321583  2942 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:26.299229  2982 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.347124 30988 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:26.344666 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.248772  2980 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.176770  2986 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.328508  2796 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.337088  2983 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.354148 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:26.358675  3048 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
I20260709 12:25:26.359766 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.360926 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.363227  3047 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:26.362262  2875 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.342338  2956 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.365439  2875 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.365713  2956 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.367879  2956 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.369640  2954 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.370316 30984 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:26.371222 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.372179 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.372697 30984 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:26.373503 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.373772 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.374451 30991 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:26.374500  2826 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.375422 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.376442 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:26.376768  2826 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.377784 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:26.378896  3045 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 12:25:26.379966 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.380028 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2441) took 1303 ms. Trace:
I20260709 12:25:26.380785 30992 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:26.381214  3032 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:26.381667 30992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:25:26.366039  2969 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.378820 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.384192 30992 raft_consensus.cc:606] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:26.372905 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:26.388727 30988 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:26.389824 30987 raft_consensus.cc:2376] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13. Re-sending same reply.
I20260709 12:25:26.389612 30992 raft_consensus.cc:614] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:26.398623 30988 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:26.349860  2942 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.410339 30987 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:26.380602 30985 rpcz_store.cc:276] 0709 12:25:25.076143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.086892 (+ 10749us) service_pool.cc:224] Handling call
0709 12:25:25.127886 (+ 40994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.595713 (+467827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.595753 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.596203 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 12:25:25.886959 (+290756us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:25.887003 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.371099 (+484096us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b54002c1354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:26.371144 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.379743 (+  8599us) raft_consensus.cc:1685] finished
0709 12:25:26.379763 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.379971 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":771736,"raft.queue_time_us":495149,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":24840,"spinlock_wait_cycles":15239552,"thread_start_us":290441,"threads_started":1,"wal-append.queue_time_us":524451}
I20260709 12:25:26.354933  2796 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.381745  3032 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:26.351361  2982 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.414443  2942 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:26.442874 30987 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:26.353489  2980 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:26.444092 31695 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1038) took 1406 ms (client timeout 1515 ms). Trace:
W20260709 12:25:26.444525 31695 rpcz_store.cc:269] 0709 12:25:25.036255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.036387 (+   132us) service_pool.cc:224] Handling call
0709 12:25:26.442790 (+1406403us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54005a5854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:26.442895 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369603328}
I20260709 12:25:26.460778 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:26.477242 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.483494 30991 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:25:26.485011 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.463243  2937 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.000s
I20260709 12:25:26.492990 31137 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:26.499682 30987 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:26.510624 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:26.511294 31138 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
W20260709 12:25:26.512475  3057 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:25:26.512598 31140 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:25:26.517978 31140 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:26.519300 31138 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:26.527673 31144 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:26.519258  2969 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:26.463536  3032 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:26.545059  3065 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:25:26.546667 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 731) took 1470 ms. Trace:
I20260709 12:25:26.547039 30986 rpcz_store.cc:276] 0709 12:25:25.076681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.087056 (+ 10375us) service_pool.cc:224] Handling call
0709 12:25:25.087346 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.594924 (+507578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.594968 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.595466 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.050852 (+455386us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b1800401200. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:26.321952 (+271100us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:26.331104 (+  9152us) log.cc:844] Serialized 14 byte log entry
0709 12:25:26.333792 (+  2688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.333860 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.546407 (+212547us) raft_consensus.cc:1685] finished
0709 12:25:26.546493 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.546608 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":819840,"prepare.queue_time_us":7141,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":35,"raft.queue_time_us":538185,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":13751,"spinlock_wait_cycles":51915648,"thread_start_us":270914,"threads_started":1,"wal-append.queue_time_us":798877}
I20260709 12:25:26.549172 30986 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:26.554809  3061 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:26.541533  2796 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.558117  2796 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.558877 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 747) took 1412 ms. Trace:
I20260709 12:25:26.559333 30983 rpcz_store.cc:276] 0709 12:25:25.146918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.262799 (+115881us) service_pool.cc:224] Handling call
0709 12:25:25.263089 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.568972 (+305883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.569018 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.569449 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:25:25.966061 (+396612us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b18003bb0c0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:26.336402 (+370341us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:26.349256 (+ 12854us) log.cc:844] Serialized 14 byte log entry
0709 12:25:26.349571 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.349691 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.558574 (+208883us) raft_consensus.cc:1685] finished
0709 12:25:26.558677 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.558799 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":464254,"prepare.queue_time_us":12566,"prepare.run_wall_time_us":24454,"raft.queue_time_us":311461,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":18491,"spinlock_wait_cycles":20850304,"thread_start_us":370174,"threads_started":1,"wal-append.queue_time_us":867995}
I20260709 12:25:26.563856 30987 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:26.543982  2649 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:26.573674 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 409738us
I20260709 12:25:26.577559 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:26.577941 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2505) took 1025 ms. Trace:
I20260709 12:25:26.578146 31695 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.580006 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:26.581014 31691 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:26.583402 31138 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:26.585101 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.586279 31058 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:25:26.588670 31033 leader_election.cc:336] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.018s (SENT)
I20260709 12:25:26.589011 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.589098 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:26.578311 31062 rpcz_store.cc:276] 0709 12:25:25.552104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.552376 (+   272us) service_pool.cc:224] Handling call
0709 12:25:25.552762 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.553003 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:26.000053 (+447050us) log.cc:844] Serialized 14 byte log entry
0709 12:25:26.577628 (+577575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.577678 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:26.577737 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.577794 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.577889 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":307202,"prepare.queue_time_us":242353,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":4618,"spinlock_wait_cycles":3379456,"thread_start_us":716694,"threads_started":2,"wal-append.queue_time_us":474181}
I20260709 12:25:26.589848 30983 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:26.596596 31698 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:26.541772  2969 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.603374 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:26.604166 31696 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:26.605340 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:26.575089  3032 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.610631 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:26.612746 31689 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:25:26.613257 31033 leader_election.cc:336] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.027s (SENT)
I20260709 12:25:26.613675 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:26.614993  2982 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:26.618719  3059 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:25:26.621258 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:26.622499  3054 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:26.623238  3054 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:26.595197  2984 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.631036 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:26.632448 31143 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:26.635771 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:26.636312 31141 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:26.636647 31695 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:26.638764 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:26.626307 30958 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:26.628468  3062 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:25:26.629534  3047 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.658851 31143 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:26.630358  3015 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.658476  2984 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.679070  3015 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.680487  2984 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:26.576522  2649 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.680959  3015 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:26.682569  2649 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.630762  2396 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:26.682843  2972 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.657675 31140 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:26.681928  2737 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.687364  2972 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:26.659789 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:26.688499 31058 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:26.612022  2869 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:26.690016  3066 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:26.689883  2869 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.692255  2869 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:26.692910 31689 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:26.635353  2982 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:26.646138  2940 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.684278  2396 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.695209  3047 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.696133 30984 raft_consensus.cc:2368] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12.
I20260709 12:25:26.642953  2941 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:26.662511 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:26.687261  2737 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.697091  2940 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.707084  3015 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.717461  3015 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.718643  2939 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.697088  2396 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:26.721102  3015 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:26.697345  3047 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:26.697579  2941 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:26.659250  2973 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:26.715447 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:26.721508  2939 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:26.749963  2737 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.755846  2969 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:26.758529  2941 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.728650  2973 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.736163  3014 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:26.767953  3014 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:26.768235  2737 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.768966  2737 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:26.771620  2737 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:26.787508 31692 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
W20260709 12:25:26.847102 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.040539  2942 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.000s
I20260709 12:25:27.030617  2649 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.042758  2942 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.043498  2649 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.045082  2942 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.031279  2436 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.046912  2436 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.031159  2860 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.050923  2860 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.053750  2860 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.025244  2983 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.075067  2983 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.077208  2983 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.108410  2649 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.110802  2436 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.085142  2973 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:27.113615  3064 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.625s	user 0.001s	sys 0.001s
I20260709 12:25:27.123873  2986 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.124289  3072 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.124497  2986 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.124866  3072 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.126988  2986 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.127583  3072 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:27.129280  3069 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.378s	user 0.000s	sys 0.001s
W20260709 12:25:27.141089 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.009s	sys 0.005s
W20260709 12:25:27.141425 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.009s	sys 0.005s
W20260709 12:25:27.141899 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 528630us
W20260709 12:25:27.142647 31033 leader_election.cc:336] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.053s (SENT)
I20260709 12:25:27.143460  2986 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.144053  2986 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.143007 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:27.144871  3072 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.147023  2984 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:27.147060  3072 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:27.148162 31692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 966) took 1518 ms (client timeout 1657 ms). Trace:
W20260709 12:25:27.148557 31692 rpcz_store.cc:269] 0709 12:25:25.630035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.460500 (+830465us) service_pool.cc:224] Handling call
0709 12:25:27.147993 (+687493us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b540011b7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:27.148103 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722871168}
I20260709 12:25:27.148885  2984 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.149371  2869 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:27.149778 31033 leader_election.cc:400] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11.
I20260709 12:25:27.149448  2984 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.151834  2984 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.151801  2986 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.149334  3072 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.153982  2984 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:27.154666  2980 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.154482  2869 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:27.155130  2986 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:25:27.157980  2869 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:27.158519  2984 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:27.159281  2984 consensus_queue.cc:1243] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:27.159716  2892 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:27.160079  2892 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:27.160921 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:27.160830  2892 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.161810  2892 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.162117  3072 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:27.162664  3072 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:27.163530  3072 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.164491 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:27.165354  2867 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:27.166834 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:27.167909 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:27.168577  2867 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:27.169203  2867 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:27.170267 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:27.171187  2867 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:27.171555  2867 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.171885  2867 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.172895 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:27.173657  2986 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:27.176568 31139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.010s	sys 0.022s
W20260709 12:25:27.176921 31139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.010s	sys 0.022s
I20260709 12:25:27.178572  2740 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.179095  2740 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.174773 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:27.181144  2740 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.183790 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:27.184809  2986 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:27.188498  2986 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:27.189551  2986 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.183737  2869 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:27.193100  2867 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.190994  2869 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.193962  3080 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:27.194731 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:27.195355  3080 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:27.194566  2869 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.195977  3080 consensus_queue.cc:1243] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:27.197077  3080 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:27.197520  3080 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:27.197685 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:27.199863  2867 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.198367  3080 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:27.209169 31058 consensus_meta.cc:287] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.520s	user 0.015s	sys 0.004s
I20260709 12:25:27.209697 31058 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:25:27.209975 31695 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:27.210754 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:27.210990 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.211524 31058 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:27.212414 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.213248 31058 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:27.213356 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:27.213934 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.214408  3015 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:27.214524 31058 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:27.214871  3015 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.215273  3015 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:27.215590  2930 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.220737 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2042) took 1177 ms. Trace:
I20260709 12:25:27.224025 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.198114  2869 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.224331  3015 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.221194 31058 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:27.237684 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.223735 31137 rpcz_store.cc:276] 0709 12:25:26.043337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.043440 (+   103us) service_pool.cc:224] Handling call
0709 12:25:27.220535 (+1177095us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54001e9bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:27.220663 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966269824}
I20260709 12:25:27.236282 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.240326 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.241490 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.237900 31106 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:27.236634 31033 leader_election.cc:336] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.482s (SENT)
I20260709 12:25:27.243110  3014 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:27.243690  3014 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:27.244784  3014 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.252213  3080 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.256291  3080 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.259635  3080 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.260988  3080 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.262881  3081 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.263435  3081 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.265352  3081 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.265727 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:27.261538  3080 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.267234 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.267977  3080 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.266040  3081 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:25:27.269160  3081 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:27.269491  3081 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:27.268422 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.269805 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:27.270501 31139 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:27.270759 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.271353  3081 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.271932 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.272589  3081 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.272593 31143 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:27.276937  3086 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:27.277195 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.277776 31143 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:27.277463  3086 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.278978 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.279469 31143 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:27.280371 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.280705 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.281263 31137 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:27.282879 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.284163 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.333551 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:27.382002 31065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.009s	sys 0.010s
W20260709 12:25:27.382465 31065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.010s	sys 0.010s
I20260709 12:25:27.383273 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1079) took 1249 ms. Trace:
I20260709 12:25:27.383612 31065 rpcz_store.cc:276] 0709 12:25:26.134239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.134416 (+   177us) service_pool.cc:224] Handling call
0709 12:25:26.134799 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.595455 (+460656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.595501 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.596094 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.382854 (+786760us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:27.382897 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.382994 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.383044 (+    50us) raft_consensus.cc:1685] finished
0709 12:25:27.383055 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.383196 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":460186,"raft.queue_time_us":474028,"spinlock_wait_cycles":9952640,"thread_start_us":731472,"threads_started":1,"wal-append.queue_time_us":498576}
W20260709 12:25:27.409148 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 754947us
I20260709 12:25:27.409876 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:27.410980 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:27.412348  3088 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.412912  3088 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.415153  3088 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.415691  3088 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:27.416165  3088 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:27.416838 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2203) took 1410 ms. Trace:
I20260709 12:25:27.417004  3088 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.417420 30987 rpcz_store.cc:276] 0709 12:25:26.006467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.534624 (+528157us) service_pool.cc:224] Handling call
0709 12:25:26.543247 (+  8623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.564224 (+ 20977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.416528 (+852304us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b54002c2254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:27.416763 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47010176}
I20260709 12:25:27.423336  3089 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:27.425055  3089 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:27.425505 30987 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:27.426473 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2453) took 1417 ms. Trace:
I20260709 12:25:27.425494  3089 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:27.426884 30987 rpcz_store.cc:276] 0709 12:25:26.009178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.424785 (+1415607us) service_pool.cc:224] Handling call
0709 12:25:27.425131 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.426031 (+   900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.426403 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:27.428061  3089 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.428735 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2454) took 1419 ms. Trace:
I20260709 12:25:27.429215 30987 rpcz_store.cc:276] 0709 12:25:26.009446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.427769 (+1418323us) service_pool.cc:224] Handling call
0709 12:25:27.428070 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.428409 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.428441 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.428483 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:27.428528 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.428581 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.428684 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:27.431758  3091 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:27.435853  3093 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:27.437732  3093 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.438195  3093 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:27.438980 30986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.007s	sys 0.013s
W20260709 12:25:27.439373 30986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.007s	sys 0.013s
I20260709 12:25:27.440117 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2447) took 1432 ms. Trace:
I20260709 12:25:27.445116  3093 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.444386 30986 rpcz_store.cc:276] 0709 12:25:26.007543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.549838 (+542295us) service_pool.cc:224] Handling call
0709 12:25:26.550092 (+   254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.578621 (+ 28529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.578649 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.579077 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.439735 (+860658us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:27.439779 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.439854 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.439913 (+    59us) raft_consensus.cc:1685] finished
0709 12:25:27.439928 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.440065 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8691,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":3557,"thread_start_us":860362,"threads_started":1,"wal-append.queue_time_us":175544}
I20260709 12:25:27.438958  3088 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.448379  3088 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.445962  3093 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.451846  3093 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.453027  3088 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.455354  3093 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.459165  3088 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.463021  3088 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.464668 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:27.465828  3093 raft_consensus.cc:2764] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:27.467100 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:27.467638 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2458) took 1419 ms. Trace:
I20260709 12:25:27.468024 30986 rpcz_store.cc:276] 0709 12:25:26.048358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.446087 (+1397729us) service_pool.cc:224] Handling call
0709 12:25:27.446444 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.446778 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:27.446809 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.447343 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.465673 (+ 18330us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:27.465712 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.465773 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.467365 (+  1592us) raft_consensus.cc:1685] finished
0709 12:25:27.467381 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.467579 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12464,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":17934,"threads_started":1,"wal-append.queue_time_us":17971}
I20260709 12:25:27.468604 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:27.470183 30986 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:27.470175 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:27.470968 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2204) took 1137 ms. Trace:
I20260709 12:25:27.471259 30986 rpcz_store.cc:276] 0709 12:25:26.333542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.469683 (+1136141us) service_pool.cc:224] Handling call
0709 12:25:27.469945 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.470768 (+   823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.470930 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:27.467828  3093 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:27.477919  2980 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.472055  3091 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:27.479476 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:27.480780  3092 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.482046  3093 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.483031  3093 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.483476  3093 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.483112  3092 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.488567 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2176) took 1508 ms. Trace:
I20260709 12:25:27.488862  2867 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.488931 30984 rpcz_store.cc:276] 0709 12:25:25.980484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.373262 (+392778us) service_pool.cc:224] Handling call
0709 12:25:27.488372 (+1115110us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b540029b8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:27.488512 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740323200}
I20260709 12:25:27.489574  2867 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.490468 30985 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:27.490268 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.491631 30986 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:27.493356 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.483160  3098 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.494349  3098 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.491689  3095 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.495270  3092 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.495482  2867 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.496903  3095 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.496173 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.500013 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2455) took 1490 ms. Trace:
I20260709 12:25:27.492300  3088 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.500471 30987 rpcz_store.cc:276] 0709 12:25:26.009726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.430172 (+1420446us) service_pool.cc:224] Handling call
0709 12:25:27.430490 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.466969 (+ 36479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.467002 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.467503 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.480495 (+ 12992us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:27.491358 (+ 10863us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.491542 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.491609 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.499819 (+  8210us) raft_consensus.cc:1685] finished
0709 12:25:27.499841 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.499959 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16655,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":33,"raft.queue_time_us":15107,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":1429,"thread_start_us":58109,"threads_started":3,"wal-append.queue_time_us":29918}
I20260709 12:25:27.501546  3093 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.501823 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.502473 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.503131 30987 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:27.503165 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.503800 30986 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:27.503930 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.504566 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:27.504663 30987 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:27.505264 30986 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.507683 30987 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:27.237207  2986 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.509501  2986 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.513463  2986 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.503978  3093 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.518805 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.522585 30987 pending_rounds.cc:85] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 5
I20260709 12:25:27.522905 30987 pending_rounds.cc:107] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:25:27.523155 30987 raft_consensus.cc:2889] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.524004 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:27.525319 30986 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:27.527806 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2451) took 1519 ms. Trace:
I20260709 12:25:27.528187 30983 rpcz_store.cc:276] 0709 12:25:26.008648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.590694 (+582046us) service_pool.cc:224] Handling call
0709 12:25:26.590941 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.651438 (+ 60497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.651467 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.651510 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:26.651557 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.527586 (+876029us) spinlock_profiling.cc:243] Waited 60.3 ms on lock 0x7b54003462d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:27.527634 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.527751 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138701184}
W20260709 12:25:27.525460 31138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.009s	sys 0.006s
W20260709 12:25:27.529451 31138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.010s	sys 0.006s
I20260709 12:25:27.536194  2989 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.536932 31059 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:27.536724  2989 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.539599 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.540170  2989 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.540187 30983 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:27.543377 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.546353  3014 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:27.550151  3015 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.552090 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2200) took 1556 ms. Trace:
I20260709 12:25:27.552487 30991 rpcz_store.cc:276] 0709 12:25:25.995508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.376244 (+380736us) service_pool.cc:224] Handling call
0709 12:25:27.551880 (+1175636us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540022e2d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:27.552030 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244339072}
I20260709 12:25:27.558502 30992 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:27.560218 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2199) took 1565 ms. Trace:
I20260709 12:25:27.560591 30992 rpcz_store.cc:276] 0709 12:25:25.995240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.443145 (+447905us) service_pool.cc:224] Handling call
0709 12:25:27.558309 (+1115164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.558909 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.560018 (+  1109us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540022e2d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:27.560155 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2519518976}
I20260709 12:25:27.569762  3093 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.570312  3093 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.572374  3093 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.577667  3088 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.578140  3088 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.579759 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.580042  3088 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.580358 31059 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:27.603235  3075 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.492s	user 0.003s	sys 0.000s
I20260709 12:25:27.604359 30983 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:25:27.602130 31141 consensus_meta.cc:287] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.965s	user 0.014s	sys 0.002s
I20260709 12:25:27.606689 31141 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
W20260709 12:25:27.612845 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.616163 30983 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:27.616231 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:27.616817 30986 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.619310  2737 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.619779  2737 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.621696  2737 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:27.623062  2737 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:27.624051  3089 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:27.624123  2792 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:27.624547  3089 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:27.622056 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:27.625360  3089 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.626914  2649 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:27.627332  2649 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.627669  2649 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:27.627952 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.626431 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:27.631994 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:27.632990 30986 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:27.633514 30983 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:27.636391 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.634793 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.638203 30983 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:27.639240 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.630270  2892 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.640714  2892 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.639904 30983 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 5.
I20260709 12:25:27.645963 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:27.647051 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.642706  2892 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.650782 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:27.651288 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.653605 30986 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:27.655103 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.655683 30986 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:27.651842 30983 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:27.656919 31141 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:27.649605  2792 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:27.660189  2792 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.660563  2792 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:27.663735  2737 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:27.664294  2737 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:27.665138  2737 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.637099  2989 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.666482  2989 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.668330 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:27.668501  2989 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.669313 31059 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:27.652094  3089 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:27.673352  3068 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:27.674974  3068 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:27.676192 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2181) took 1694 ms. Trace:
I20260709 12:25:27.675954  2649 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.676546 30988 rpcz_store.cc:276] 0709 12:25:25.982189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.412420 (+430231us) service_pool.cc:224] Handling call
0709 12:25:26.412735 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.413128 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.413164 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.442137 (+ 28973us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.668549 (+226412us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:26.668603 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.668689 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.675994 (+1007305us) raft_consensus.cc:1685] finished
0709 12:25:27.676018 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.676139 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":653,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":21664,"thread_start_us":184131,"threads_started":1,"wal-append.queue_time_us":266220}
I20260709 12:25:27.678354  2796 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.682577 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2183) took 1699 ms. Trace:
I20260709 12:25:27.694845 30985 rpcz_store.cc:276] 0709 12:25:25.982690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.439322 (+456632us) service_pool.cc:224] Handling call
0709 12:25:27.490306 (+1050984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.491187 (+   881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.682399 (+191212us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540029b8d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:27.682520 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2401687808}
I20260709 12:25:27.649811  2941 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:27.683218 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:27.687949  3086 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.698338 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.696751 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:27.704550 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:27.702425  2941 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.709462  2941 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.710749  2649 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:27.716583  3063 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.196s	user 0.000s	sys 0.003s
I20260709 12:25:27.703431  3014 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:27.717387  3014 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:27.718250  3014 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.674000  2792 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.726714 30991 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
W20260709 12:25:27.747972 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.749366 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.749954 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.750857 31062 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:27.751135 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.752130 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.752511 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.751754 31059 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:27.753340 31058 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:27.752703 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.754030 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.754268 31060 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:27.755010 31063 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:27.759493 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.760068 31061 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:27.763928 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:27.765964 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:27.768797 30955 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:27.754487  2869 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.770653 31061 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.772706 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:27.773592  3088 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:27.774077  3088 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.774501  3088 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:27.774753  2869 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.773669 31063 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:27.777462  2792 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:27.771504  3093 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:27.779567  2889 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:27.776224  3092 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:27.781397  3092 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.781777  3092 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.769959  3089 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:27.786556  3089 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.786952  3089 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.788479  2984 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:27.788968  2984 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:27.790575  3093 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:27.791033  3093 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.797832 31062 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:25:27.798123 31058 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:25:27.799065 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.799584 31058 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:27.798205 31062 pending_rounds.cc:85] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 9
I20260709 12:25:27.802377 31062 pending_rounds.cc:107] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:27.802610 31062 raft_consensus.cc:2889] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.804497  3088 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.802040 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2448) took 1794 ms. Trace:
I20260709 12:25:27.806361 30989 rpcz_store.cc:276] 0709 12:25:26.007835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.553456 (+545621us) service_pool.cc:224] Handling call
0709 12:25:26.553721 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.626750 (+ 73029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.626779 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.626819 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:26.626859 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.801836 (+1174977us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b5400022bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:27.801883 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.801985 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167551872}
I20260709 12:25:27.809136  2869 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:27.811036  3032 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.811513  3032 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.813380  3032 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:27.814541 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.818501 31061 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:27.820458 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.844341 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.771039  3108 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:27.847693 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.852923 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:27.853554 31060 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:27.842656  2737 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:27.857331 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.858022 31060 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.860694  3115 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:27.847513  3108 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:27.856853  2737 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.882761 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:27.891590 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 184346us
I20260709 12:25:27.864861  2941 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.892673  3108 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:27.893508  2737 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:27.893718 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.881064  2939 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.880059  3088 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:27.902411 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 104284us
I20260709 12:25:27.893898  2792 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:27.927299 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.928073 30985 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:25:27.911337 31060 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:27.928833 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.929541 30988 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:27.932857 30983 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:27.933164 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:27.936527 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 118800us
I20260709 12:25:27.938092 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:27.938638 31138 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.939692 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:27.940838 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.941958 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:27.943483 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.944579 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.947535 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:27.953764 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.954550 30988 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:27.956804 31144 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:27.962245 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:27.966396 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 60732.3us
I20260709 12:25:27.967077 31664 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:27.968649  3116 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:25:27.970414  2824 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:27.972010 30992 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:27.974051 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:27.976961 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.978349 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:27.979018 31144 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.981956 31136 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:27.983613 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.984241 30989 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:27.977710 31143 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:27.986999 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:27.987577 30984 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:27.992389 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:27.992522 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:27.993121 31140 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:27.994045 31136 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:27.994761 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:27.997382 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 63230.3us
W20260709 12:25:27.967947 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 124203us
I20260709 12:25:27.933491  3047 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:28.004808 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:28.012324 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:27.983580  2989 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:28.020227  2824 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:28.020401 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.039115 31144 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:28.020834  2973 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.022390  3091 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:28.021893  2737 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.022888  2942 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:28.023090 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:28.028162 31138 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:28.038281 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.045620 31691 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:27.930724  2939 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.050184 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:27.984476  3072 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:25:27.984694  3115 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:25:28.042567 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:28.046759 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:28.055024 31695 raft_consensus.cc:606] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:27.984350  2869 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:25:28.046752 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:28.047564 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:27.996848  2984 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.016798  2649 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.008026  3108 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.038589 31060 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:28.051009 31693 raft_consensus.cc:2376] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14. Re-sending same reply.
I20260709 12:25:28.040002  2989 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.062427  2989 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.999946  3124 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:28.045321  3015 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.044247  2930 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:25:28.061115 31692 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:28.048012  2973 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.047276  3127 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.059887 31141 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.009256  3089 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.092406  3089 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.094400 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:28.096609  3089 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:28.097353  3089 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.099851  3089 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.104046 31141 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
W20260709 12:25:28.077930 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 94063.3us
I20260709 12:25:28.113101 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:28.040309  3047 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.066205  3091 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:28.064955  2649 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.065742  3124 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:28.120669 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 97651.6us
I20260709 12:25:28.064843  3126 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.120033  3121 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.127238  3126 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.127588  3121 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.129320  3126 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:28.123170 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.129607  3121 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.066336  2860 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:25:28.125490  3091 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.131291  3091 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.134308  2649 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.135630  3089 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.136097  3089 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.138072  3089 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.139866  3047 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.146365  3124 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.147145  2939 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.148205  3129 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.148687  3129 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.125504  2941 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.150784  3129 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:28.153035 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:28.153676 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.126683  2915 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.166555  2915 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.167110 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:28.167879 30992 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:28.168510  2915 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.110930  3072 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:28.175148 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:28.175948 31063 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:28.120865  3068 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.193082 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.192796  3072 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.195439  3072 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.199213  2737 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:28.203364 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:28.204653 30990 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:28.205912 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2457) took 2195 ms. Trace:
I20260709 12:25:28.206321 30990 rpcz_store.cc:276] 0709 12:25:26.011247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.378629 (+367382us) service_pool.cc:224] Handling call
0709 12:25:28.205701 (+1827072us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54000762d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:28.205850 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4174201088}
I20260709 12:25:28.207330  3093 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.186141  3098 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.211436 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.212342 31689 raft_consensus.cc:2368] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10.
I20260709 12:25:28.084422  3015 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.215261  3015 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:28.216852 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:28.217693 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:28.220165  2986 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:28.220623  2986 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:28.221052  2986 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:28.223352  3072 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.223858  3072 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:28.224289  3072 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.231055  2979 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:28.234843  2984 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:28.236613 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:28.237236  2969 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:28.237463  2984 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:28.237892  2984 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:28.238425  2984 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.240700 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:28.241240 31693 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.241657 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.242257 31690 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:28.243134 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.243993 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.244544 31697 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:28.246933 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.248183 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.186995 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:28.188223 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.189138  3095 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.190591  2973 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.109651  3127 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.161191  3089 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:28.191821 31698 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:28.255300 31064 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:28.253939 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:28.258978 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 204540us
I20260709 12:25:28.262180 31063 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:28.209998  3093 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.274816 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.276360 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.277572 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.280025 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:28.281950 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 283932us
I20260709 12:25:28.311492 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:28.288762 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.294499 31143 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:28.295369 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.313475 31141 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:28.314275 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:28.295809 31138 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.296716 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.297703 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.315323 31135 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:28.298766 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.300127 31136 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [8.5-11.6]
I20260709 12:25:28.300159 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.300804  2869 consensus_queue.cc:1243] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 10.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:28.301041 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.298167  2989 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.302255 31142 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [9.4-11.5]
I20260709 12:25:28.325573 31693 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:28.316898 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:28.326932 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.317965 31689 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:28.197715  3092 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.322872 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.335464 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:28.323534 31697 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:28.323828 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.336628 31136 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:25:28.336747 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.324486 31141 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.337394 31698 raft_consensus.cc:2368] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7.
I20260709 12:25:28.333837 31138 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:28.339840 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:28.340261 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:28.340953 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:28.237926  2969 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.344810 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.355722 31142 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:25:28.345762 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 65853.7us
I20260709 12:25:28.346705 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.357102 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:28.357347 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:28.348063 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.358280 31136 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:28.345795 31138 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:28.358842 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:28.346165 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.360743 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.362027 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.347525 30990 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:28.367583 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:28.368270 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:28.368561 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:28.369114 30989 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:28.369481 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:28.367610 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.370754 31136 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:28.370832 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:28.371495 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 100707us
I20260709 12:25:28.372162 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.374447 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:28.375164 31138 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.373873 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:28.376513 31141 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.368778 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:28.377794 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.378121 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:28.378382 31690 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:28.268199  2979 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 12:25:28.379289 30991 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.308729  2878 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:25:28.343295  2973 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.342051  3086 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:28.395473 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:28.367632  2796 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.319847  3089 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.344274  3130 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:28.406744 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:28.385288  2805 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:28.366827  3131 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.413983 30989 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:28.351611  2869 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:25:28.384197  2930 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:28.387461  2939 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:28.443552 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 115707us
I20260709 12:25:28.384302  3093 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:28.434319 31138 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:28.447340 31143 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:28.434993 31141 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:28.353315  2824 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:25:28.454308  2805 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.454787  2805 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.455484  2824 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:28.455860  2824 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.456174  2824 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:28.456723  2805 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.457422 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.458595 31694 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:28.458643 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.459836 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.460956 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.461072 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:28.461417  3075 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:28.407754  3141 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:28.446195  2939 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.462817  2939 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:28.406152  3086 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.406414  2796 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:28.465427  2796 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.467037  3140 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:28.467490  3140 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.467840  3140 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.349279  3122 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:25:28.470487  3129 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:28.471071  3129 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.440145  3131 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:28.473325 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:28.473534  3131 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.476713  3123 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:28.393411  2942 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:28.448629 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08003aee00 after lost signal to thread 29848
I20260709 12:25:28.387126  3128 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:25:28.457370  3095 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:25:28.446672 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.482311 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:28.483263 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:28.451350  3092 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.492961 31694 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:28.496047 31690 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:28.496573 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:28.496764 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52057.3us
I20260709 12:25:28.516947 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:28.555483 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:28.477253  3123 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:28.407073  2973 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.496524  3141 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:28.542503  2930 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:28.592864 31143 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:28.598913 31141 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:28.600323 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.632061 31138 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:28.603744 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:28.612078 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08000879e0 after lost signal to thread 30924
W20260709 12:25:28.593722 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 46775.8us
W20260709 12:25:28.637203 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08001e93c0 after lost signal to thread 31016
W20260709 12:25:28.637566 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4e40 after lost signal to thread 31017
I20260709 12:25:28.638908 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:28.556432  2436 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:28.648411  2892 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:28.648912  2892 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:28.649892  2892 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.651150  2892 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.651228 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.652109 31141 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:28.655172  3129 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:28.656239  3091 consensus_meta.cc:287] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.000s
I20260709 12:25:28.602000  2942 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:28.657205  2984 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.658303  2942 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.659636  2984 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.661091  3141 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.591886  3132 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 12:25:28.665359 31063 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:28.634719 31142 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:28.555202  3093 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:28.591931  3089 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:28.681075 31165 debug-util.cc:398] Leaking SignalData structure 0x7b080031d540 after lost signal to thread 31093
W20260709 12:25:28.681499 31165 debug-util.cc:398] Leaking SignalData structure 0x7b0800402ae0 after lost signal to thread 31094
I20260709 12:25:28.686861 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:28.690618 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.702358 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08003c0000 after lost signal to thread 31168
W20260709 12:25:28.702764 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08003c8ea0 after lost signal to thread 31169
W20260709 12:25:28.704164 31165 debug-util.cc:398] Leaking SignalData structure 0x7b0800315e60 after lost signal to thread 31722
W20260709 12:25:28.704489 31165 debug-util.cc:398] Leaking SignalData structure 0x7b0800362000 after lost signal to thread 31723
W20260709 12:25:28.704766 31165 debug-util.cc:398] Leaking SignalData structure 0x7b0800320ac0 after lost signal to thread 2396
I20260709 12:25:28.731541 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.729962  2860 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.734441  2930 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.734784  2860 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.736868  2860 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.740880  2939 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.742209 31140 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:28.743178  3127 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:28.743995  2939 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.744820  2989 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.747103  3093 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:28.747866  2649 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.001s
I20260709 12:25:28.748620  2649 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.750151 31692 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:28.751413  2973 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:28.752195  3123 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:28.755393  3124 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.000s
I20260709 12:25:28.755897  3124 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.757225 31691 raft_consensus.cc:2368] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10.
I20260709 12:25:28.673259  2436 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:28.759199  2436 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.766575  3086 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:28.768232 31690 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:28.777402 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 304137us
W20260709 12:25:28.786211 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 417461us
I20260709 12:25:28.788236 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:28.656647  3091 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:28.794185 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:28.796296 31060 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.009s	sys 0.014s
W20260709 12:25:28.771658  2986 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.000s
W20260709 12:25:28.806808 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 411394us
I20260709 12:25:28.807878 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:28.809386 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:28.795876 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:28.774307  3127 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:28.805392 31060 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.009s	sys 0.014s
I20260709 12:25:28.856122  3127 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.819514 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:28.857201  3143 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.261s	user 0.006s	sys 0.001s
I20260709 12:25:28.822055 30988 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:28.858673 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 62908.4us
W20260709 12:25:28.860600 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 51303.6us
I20260709 12:25:28.872373 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:28.879065 31065 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:28.880384 31063 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.823258 31109 leader_election.cc:400] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10.
W20260709 12:25:28.823484 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.796828  3124 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:28.883121  3089 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:28.883337 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:28.796504  2979 consensus_queue.cc:1243] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:28.814330  2986 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:28.846731 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:28.819924  3092 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:28.820029  2939 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.886976  2986 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:28.887780 31109 leader_election.cc:403] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.893983 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:28.903134 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:28.914059 30988 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:28.916064 31142 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:28.917402 30958 leader_election.cc:400] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7.
I20260709 12:25:28.919000 30958 leader_election.cc:403] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.921481 30958 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:28.925097  3089 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.877904  2915 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:28.917726 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:28.866776  3127 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:28.996881  3159 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10.
I20260709 12:25:28.925696  3088 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:25:28.878918  2973 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:28.929227  3137 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.890601  3092 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:29.000610 31144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.017s	sys 0.008s
W20260709 12:25:29.033456 31144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.017s	sys 0.008s
I20260709 12:25:28.891120  2796 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:28.998827 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 175654us
I20260709 12:25:28.997556 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:29.014648  3135 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:29.029744  3140 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.000s
I20260709 12:25:29.016332 30985 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:29.021493 31063 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:29.020704 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59170 (request call id 5) took 1005 ms. Trace:
I20260709 12:25:29.048462 31695 rpcz_store.cc:276] 0709 12:25:28.036896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.042202 (+  5306us) service_pool.cc:224] Handling call
0709 12:25:28.201701 (+159499us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b54006e1020. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb811520 00007f2eeb88e652 00007f2ef085c60a 00007f2ee9de27d4 00007f2ee9de2762 00007f2ee9de2691 00007f2ee9de260c 00007f2ee9de18c2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:28.982787 (+781086us) spinlock_profiling.cc:243] Waited 86.1 ms on lock 0x7b54006e1020. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb80f5e7 00007f2eeb88ec68 00007f2eeb88e84a 00007f2ef085c60a 00007f2ee9de27d4 00007f2ee9de2762 00007f2ee9de2691 00007f2ee9de260c 00007f2ee9de18c2 00007f2ee9bcf48c
0709 12:25:29.013443 (+ 30656us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208801408}
I20260709 12:25:29.021989 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:29.049638 30987 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:29.000339  2869 consensus_queue.cc:1243] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:29.014248  2930 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 6. Current term is 7.
W20260709 12:25:28.981346 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.009s	sys 0.026s
I20260709 12:25:29.022374  3089 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.021184  2915 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:29.054072 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.009s	sys 0.026s
W20260709 12:25:29.059314 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 684081us
I20260709 12:25:29.032963  2973 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.024416  2649 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:29.040066  3131 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
W20260709 12:25:29.062642 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 64983.1us
W20260709 12:25:29.054876  3072 consensus_meta.cc:287] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.830s	user 0.008s	sys 0.007s
I20260709 12:25:29.052994  3108 raft_consensus.cc:993] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:29.072474 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:29.060977 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:29.039695  3156 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.100438 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:29.055011  3093 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:29.083426  2986 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:29.087424 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 160399us
W20260709 12:25:29.090257 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54151.6us
I20260709 12:25:29.075198 31063 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:29.105075 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:29.058964  3089 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:29.057682  3152 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:29.077196  3115 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:29.117547 30987 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
W20260709 12:25:29.080843  2824 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.000s
I20260709 12:25:29.064417  2973 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:29.042554  3137 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.063743  2915 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:29.121964 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.073024  2805 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:29.129945 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:29.120496 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:29.107366  3091 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.115255  2649 raft_consensus.cc:1081] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:29.109560  2986 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:29.144726 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 223317us
I20260709 12:25:29.054561  3140 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.119472  3108 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:29.127010  2979 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.151052 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:29.100291  3072 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.149480 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.142696  2970 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:29.123010  2436 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:25:29.167311 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:29.079655  3131 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.135228  3001 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:29.128201  2792 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:25:29.152019  2986 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:29.153110  3160 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:29.188948 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:29.191612 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:29.194931 31142 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:29.189503 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:29.190186 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:29.195854 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:29.192170 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:29.192735 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:29.134706  2824 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.144469  2805 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.199925 31067 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:29.199698 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:29.174333  2860 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:29.195277 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.173272  3108 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:29.199481  3093 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:25:29.178421  2892 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.200896 31141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76399531471a4790a56d52a6ea052864"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:29.199236 31142 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:29.206779 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:29.155055  2869 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:25:29.206859 31141 raft_consensus.cc:606] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:29.252144 30988 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:29.194067  3160 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:29.180050  2979 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.154805  3150 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.201s
I20260709 12:25:29.127758  3148 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.229s
I20260709 12:25:29.300480 30983 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [9.4-11.5]
I20260709 12:25:29.326327 30985 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:29.360913 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:29.300513 30987 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:29.118731  3158 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.251s
I20260709 12:25:29.206146  3138 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:29.115139  3156 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.257349 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:29.199673  2970 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:29.199687  2930 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:29.199019  3092 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:29.197712  2989 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.203315  3088 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.317592  3075 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:25:29.362808  3154 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:25:29.198906  3001 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.367321  2436 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.126791  3152 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.349534  2796 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.194093  3115 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:29.324016  2892 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:29.206972  3180 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:29.257134 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:29.377777  3160 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:29.405550 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 148530us
I20260709 12:25:29.406934 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:29.409602 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 260628us
I20260709 12:25:29.387985  3148 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:29.411043 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.423561 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:29.391763  3088 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:29.399727  3173 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
W20260709 12:25:29.408746  3163 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 12:25:29.388221  2869 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:29.397239  3091 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:29.410612  2796 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:29.382906  2930 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.448877 31697 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:29.429991  2989 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:29.381577  3108 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.396765  2436 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.438340  2942 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.440268  3123 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:29.405570  2892 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.448812  2796 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.427789  2860 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:25:29.407727  3180 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:29.505429  3164 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 12:25:29.506475  3170 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 12:25:29.450007  3171 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 12:25:29.508354  3187 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:29.557068  2892 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:29.558184  3182 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:25:29.561241 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2223) took 1307 ms. Trace:
I20260709 12:25:29.508863 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 974) took 1267 ms. Trace:
W20260709 12:25:29.504294  3178 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:25:29.491117  2989 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:29.497505  3123 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.450716  3140 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:29.500798  3167 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
W20260709 12:25:29.573710 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 148942us
I20260709 12:25:29.500418  2805 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.454123  3072 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.576277 31691 rpcz_store.cc:276] 0709 12:25:28.241642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.242909 (+  1267us) service_pool.cc:224] Handling call
0709 12:25:29.504905 (+1261996us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b5400690354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:29.508802 (+  3897us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180683136}
I20260709 12:25:29.576131 31060 rpcz_store.cc:276] 0709 12:25:28.253456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.254344 (+   888us) service_pool.cc:224] Handling call
0709 12:25:28.254717 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.856192 (+601475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.856225 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.856663 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.081159 (+224496us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:29.081193 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.081256 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.561027 (+479771us) raft_consensus.cc:1685] finished
0709 12:25:29.561046 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.561184 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54946,"raft.queue_time_us":109981,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":19484,"thread_start_us":809451,"threads_started":2,"wal-append.queue_time_us":276527}
I20260709 12:25:29.458791  3091 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.508641  2878 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:29.563405  2796 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:29.575721 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 380489us
I20260709 12:25:29.452683  2970 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.554843  3131 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.627597  2969 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.451764  2869 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:29.607486 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.574110  2860 consensus_queue.cc:1243] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:25:29.508502  3137 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:29.612326  3194 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:29.612957  2930 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.617367  2915 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.617776  3192 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:29.618984  3172 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:25:29.619415  3183 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:25:29.686444  3152 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:29.625862  3153 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 12:25:29.627069  3184 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:29.688812  2983 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:29.689231  2983 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:29.689779 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2250) took 1388 ms. Trace:
I20260709 12:25:29.655160  3156 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.627163  2979 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.627763  2436 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.690589  3200 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.633694 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:29.678206  3137 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:29.691433 30985 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:29.691138  3200 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.692360  3180 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.690127 31144 rpcz_store.cc:276] 0709 12:25:28.301463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.322385 (+ 20922us) service_pool.cc:224] Handling call
0709 12:25:28.322738 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.324559 (+  1821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.324602 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.325117 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.033919 (+708802us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:29.033970 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.034054 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.689580 (+655526us) raft_consensus.cc:1685] finished
0709 12:25:29.689598 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.689728 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22364,"raft.queue_time_us":15783,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":20732,"thread_start_us":642930,"threads_started":1,"wal-append.queue_time_us":835305}
I20260709 12:25:29.693243  3200 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.652626 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.695779 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:29.697157  2979 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:29.698063  2979 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:29.698474  2979 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:29.678344  3194 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.698896  2979 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:29.700431  3194 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.670915  3201 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:29.702354  3192 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.702725  3201 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.621605  3088 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.634939  2878 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:29.704483  3192 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.704813  3201 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.670997  3158 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:29.704532  2878 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.705454  3158 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.706704  3091 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.677220  3189 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:29.707505  3189 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.709532  3189 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:29.710427  3144 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.087s	user 0.000s	sys 0.013s
I20260709 12:25:29.655661  3001 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.607249  3093 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:29.674798 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.566339  3115 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:29.662832  3166 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:25:29.677677 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 750) took 1326 ms. Trace:
I20260709 12:25:29.714638 31140 rpcz_store.cc:276] 0709 12:25:28.350979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.390684 (+ 39705us) service_pool.cc:224] Handling call
0709 12:25:28.391001 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.742667 (+351666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.677429 (+934762us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b540024a754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:29.677620 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807199616}
I20260709 12:25:29.665766  2860 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:25:29.678680 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.682336  3135 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:29.683432  3142 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.089s	user 0.012s	sys 0.005s
I20260709 12:25:29.625063 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2072) took 1280 ms. Trace:
I20260709 12:25:29.717952 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.627058  2989 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.691493  2869 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.674752 31063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:25:29.697113 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.563632  2942 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.732249  2942 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.733860  2989 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.702970 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.734356  2942 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:29.735111  3145 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.264s	user 0.005s	sys 0.007s
W20260709 12:25:29.713809  2973 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.568s	user 0.004s	sys 0.006s
I20260709 12:25:29.717556 31139 rpcz_store.cc:276] 0709 12:25:28.344655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.344753 (+    98us) service_pool.cc:224] Handling call
0709 12:25:28.345044 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.380180 (+ 35136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.380209 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.438030 (+ 57821us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b2c000e06f8. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:25:28.442948 (+  4918us) log.cc:844] Serialized 26 byte log entry
0709 12:25:28.453522 (+ 10574us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4800251958. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef07bdaa1 00007f2ef07b4b37 00007f2ef07b49ca 00007f2ef08ac26f 00007f2ef08abf37 00007f2ef08c055c 00007f2ef08c0508 00007f2ef08c0488 00007f2ef08c0443 00007f2ef08bf6ec 00007f2eef7e9a96
0709 12:25:29.013569 (+560047us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:29.013708 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.013989 (+   281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.624808 (+610819us) raft_consensus.cc:1685] finished
0709 12:25:29.624830 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.625003 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61091,"raft.queue_time_us":53425,"raft.run_cpu_time_us":2163,"raft.run_wall_time_us":47973,"spinlock_wait_cycles":58969344,"thread_start_us":457090,"threads_started":1,"wal-append.queue_time_us":615095}
I20260709 12:25:29.723929 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.726830 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.728219 31063 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:29.720985  3135 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:29.672490 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.735146 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.741942 31063 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:29.742233 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:29.702103  3089 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.005s
I20260709 12:25:29.631238  3123 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:29.686335  3181 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:29.742326 31060 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:29.749845 31060 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:25:29.751885 31060 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:29.752131 31062 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:29.752828 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2219) took 1593 ms. Trace:
I20260709 12:25:29.743295 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.738173 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.747366 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:29.735191  3198 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:25:29.717888  3091 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.748514 31063 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.758738 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.755117 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:29.753302 31059 rpcz_store.cc:276] 0709 12:25:28.159142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.159319 (+   177us) service_pool.cc:224] Handling call
0709 12:25:28.159689 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.160090 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.160129 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.250086 (+ 89957us) log.cc:844] Serialized 26 byte log entry
0709 12:25:28.293483 (+ 43397us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:28.293541 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.293631 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.734433 (+1440802us) spinlock_profiling.cc:243] Waited 6.84 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:29.748253 (+ 13820us) raft_consensus.cc:1685] finished
0709 12:25:29.748273 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.752750 (+  4477us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":542,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9423,"spinlock_wait_cycles":15720832,"thread_start_us":42937,"threads_started":1,"wal-append.queue_time_us":129451}
I20260709 12:25:29.753619  3181 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.750540  2973 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.761808 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.762118  2649 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:29.762737 30991 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:25:29.763094 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:29.691512  3137 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.765666 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:29.757826  3187 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:29.769815  3189 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.771457  3181 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.777477  3158 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.750923  3089 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.777341  3187 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.690052  2983 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.779018  2983 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:29.779300  3089 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.750991  2869 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:25:29.780473  3208 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:29.781282  2869 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.783107  3068 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:29.783479  3068 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:29.783584  2869 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:29.785250 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:29.786157  3146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.188s	user 0.013s	sys 0.009s
I20260709 12:25:29.786248  3147 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:29.786672  3146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.188s	user 0.013s	sys 0.009s
I20260709 12:25:29.764005  3091 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.786785  3147 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:29.791986 31063 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.804009  3150 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:29.804461  3150 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:29.805326  3150 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:29.808063 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.815429  3068 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:29.816742  3068 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:29.819139  3179 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.664s	user 0.003s	sys 0.003s
I20260709 12:25:29.820384 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:29.824172  3150 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:29.842110 30884 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.862704 30884 catalog_manager.cc:5697] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.865260 30884 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.867619 30884 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.869890 30884 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:29.725244  3093 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:29.880214 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 107381us
I20260709 12:25:29.763242  2942 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.762522  2649 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.021189  2649 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.022168  2649 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.022445 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1109) took 1160 ms. Trace:
I20260709 12:25:30.022801 31067 rpcz_store.cc:276] 0709 12:25:28.861634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.868327 (+  6693us) service_pool.cc:224] Handling call
0709 12:25:28.925742 (+ 57415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.202370 (+276628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.006697 (+804327us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54003971d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:30.022387 (+ 15690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627732224}
I20260709 12:25:29.886690  2869 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:30.026669 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2249) took 1725 ms. Trace:
I20260709 12:25:30.027101 31135 rpcz_store.cc:276] 0709 12:25:28.301187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.338334 (+ 37147us) service_pool.cc:224] Handling call
0709 12:25:28.338618 (+   284us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:28.408924 (+ 70306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.408964 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:28.409539 (+   575us) log.cc:844] Serialized 52 byte log entry
0709 12:25:28.878330 (+468791us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:28.878373 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.009683 (+1131310us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b540001c054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:30.026323 (+ 16640us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.026374 (+    51us) raft_consensus.cc:1685] finished
0709 12:25:30.026387 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.026595 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85428,"raft.queue_time_us":62682,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":27700,"spinlock_wait_cycles":9630080,"thread_start_us":402838,"threads_started":1,"wal-append.queue_time_us":588757}
I20260709 12:25:30.029542 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2251) took 1727 ms. Trace:
W20260709 12:25:29.888283 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 103097us
I20260709 12:25:29.763036  2969 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.029922 31137 rpcz_store.cc:276] 0709 12:25:28.301728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.323485 (+ 21757us) service_pool.cc:224] Handling call
0709 12:25:28.364137 (+ 40652us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:28.380659 (+ 16522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.380687 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:28.657805 (+277118us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b2c000e06f8. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:25:28.658496 (+   691us) log.cc:844] Serialized 52 byte log entry
0709 12:25:28.880181 (+221685us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:28.926599 (+ 46418us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.029084 (+1102485us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b18002afa60. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:25:30.029251 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.029319 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.029357 (+    38us) raft_consensus.cc:1685] finished
0709 12:25:30.029370 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.029486 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125595,"prepare.queue_time_us":72524,"prepare.run_cpu_time_us":6874,"prepare.run_wall_time_us":14661,"raft.queue_time_us":41338,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":27283,"spinlock_wait_cycles":19689728,"thread_start_us":221267,"threads_started":1,"wal-append.queue_time_us":401032}
I20260709 12:25:30.033699  2824 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.781338  2973 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.898814  3136 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.882812  3152 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.909757  2915 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:29.896292  3147 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:30.035581  3147 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:29.911576  2986 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.001s
W20260709 12:25:30.038367 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 1.00241e+06us
I20260709 12:25:30.038468  2986 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.040444  2986 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.890810  3181 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:25:30.041711 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:30.043188  3141 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:30.043540  3127 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:30.043550  3202 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:30.046757  3202 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.047559  3202 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.048233 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 751) took 1155 ms. Trace:
I20260709 12:25:30.048759  3202 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:30.049304 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 743) took 1704 ms. Trace:
I20260709 12:25:30.050863 31106 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:30.055830  3128 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.056217  3128 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:30.056542  3134 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:30.057606  3121 raft_consensus.cc:993] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:30.050894 31067 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:30.059417 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:30.061169 31063 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
I20260709 12:25:30.061590 31063 pending_rounds.cc:85] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 4
I20260709 12:25:30.061847 31063 pending_rounds.cc:107] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 12:25:30.062124 31063 raft_consensus.cc:2889] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.063376 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.064118 31059 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:25:29.919781  3138 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.000s
I20260709 12:25:29.936715  3135 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.937464  3091 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:29.943212  3123 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.075464 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.076036 31061 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:30.076067 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.076370  3123 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:29.932988  3160 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
I20260709 12:25:30.076762 31144 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:30.077313 31140 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:30.077128  3160 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.078300 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.079104  3160 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.079895 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.080324 31140 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.969004 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:30.081522 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.081557 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.082818 31141 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:29.950011  2796 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:29.970602 30990 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:29.971565 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.083881 31141 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:25:29.975904 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.083941 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:29.971815 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.084928 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:30.085501 31061 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:30.083631 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:29.996336  2878 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.093654 31661 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:29.968225  3209 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:30.110464  3209 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.112730  3209 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.020107 30958 leader_election.cc:400] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10.
I20260709 12:25:30.135932 30958 leader_election.cc:403] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:30.019634  3093 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.049695 31136 rpcz_store.cc:276] 0709 12:25:28.344916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.386651 (+ 41735us) service_pool.cc:224] Handling call
0709 12:25:28.386884 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.387085 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:28.387322 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.148060 (+760738us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b1800024700. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:25:29.182168 (+ 34108us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.048992 (+866824us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:30.049051 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:30.049095 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.049154 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.049252 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":888926,"prepare.queue_time_us":2661,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":8579,"raft.queue_time_us":82638,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":6654208,"thread_start_us":33692,"threads_started":1}
I20260709 12:25:30.136556 31067 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:30.138201 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:30.139533 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.140708 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:29.925029  3137 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.065171 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.143029 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.143759 31060 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:30.144440 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.145444 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:29.913906  2979 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:30.145742 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.148178  2979 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.148562 30986 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:30.148792 31062 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:30.149757 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:30.150063 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.150385 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.150719 30990 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 7.
I20260709 12:25:30.151136 30986 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:30.151727 31062 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:30.069651 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.153115 31068 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:30.153470  3137 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:25:29.997341 30886 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 319) took 29567 ms (client timeout 15000 ms). Trace:
I20260709 12:25:30.048624 31143 rpcz_store.cc:276] 0709 12:25:28.893270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.893441 (+   171us) service_pool.cc:224] Handling call
0709 12:25:28.893754 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.893943 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:29.076751 (+182808us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.047883 (+971132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:30.047938 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:30.047990 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.048059 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.048177 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":946877,"prepare.queue_time_us":109325,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":64,"raft.queue_time_us":920331,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":180431,"threads_started":2,"wal-append.queue_time_us":267448}
I20260709 12:25:30.071810 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.151813  2649 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:25:30.056556  3128 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.156513  3137 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:30.156883  3137 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.157684  3137 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.158883  3137 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:30.058081  3121 raft_consensus.cc:1081] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:30.153350 30884 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 6, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 6 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:30.074473  3135 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:30.078372  3159 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:30.165146  3159 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.166803 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:30.167443 30990 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.167845  3159 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.079053 31137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:30.168653 31137 raft_consensus.cc:606] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:30.168998 31137 raft_consensus.cc:614] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:29.981339  3148 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:30.087976 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.089376 31065 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:30.170127 31135 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.171945 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.071014  2986 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.099922 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:30.105809  3115 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:30.106807  3122 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:30.176153  2986 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:30.176529 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 76668.5us
I20260709 12:25:30.098696  3123 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:30.177361 31135 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:30.177644  3123 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.178206  2986 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.179072  2983 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:30.179450  2983 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:30.179577 31067 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:25:30.179735  3187 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:30.095244  3139 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:25:30.098748  3134 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:25:30.180305  3187 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.180722  3134 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:30.100453  3014 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:25:30.181239  3187 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.098555  3149 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:25:30.178108  3123 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.181167  3139 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:30.181098  3134 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:30.094990  2915 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:25:30.183265  3187 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.183543  2915 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:30.183936  2915 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.183688  3134 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.184922  3134 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:30.184738  2915 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.094563  3138 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:30.100309  2796 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.091616 31063 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:30.118541 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.121641  2969 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:30.095624 31139 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:30.181841  3014 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:30.093151  3133 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:30.186828  3014 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:30.187094  3133 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:30.187870 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 167891us
I20260709 12:25:30.187629  3014 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:30.188719  3014 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.188830 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:30.189146  3133 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.186206 31064 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:30.121613 31140 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:25:30.190382 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.191560  3181 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:30.191792  3122 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:30.096252 31061 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:30.134176  3147 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:30.134963 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:29.982344 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1113) took 1102 ms. Trace:
I20260709 12:25:30.073598  2860 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:30.193290  2915 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:30.205534  2860 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.206750  2860 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:30.207763 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.208137  2860 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:30.208630 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.209241 31137 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:30.209301  2824 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:25:30.201486 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 77166.7us
I20260709 12:25:30.210037 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.210922 31143 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:30.211102 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:30.211752 31061 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:25:30.174952  2649 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:30.215723 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.216403 31137 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
W20260709 12:25:30.217962 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.179919 31067 pending_rounds.cc:85] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 11
I20260709 12:25:30.002674 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 775) took 1663 ms. Trace:
I20260709 12:25:30.218703 31067 pending_rounds.cc:107] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:25:30.219208 31067 raft_consensus.cc:2889] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.218709 30984 rpcz_store.cc:276] 0709 12:25:28.338752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.340658 (+  1906us) service_pool.cc:224] Handling call
0709 12:25:28.368134 (+ 27476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.673925 (+305791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.673965 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.680626 (+  6661us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.028895 (+348269us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b18003bafa0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:29.388837 (+359942us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:29.388862 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.002367 (+613505us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54003458d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:30.002416 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.002465 (+    49us) raft_consensus.cc:1685] finished
0709 12:25:30.002479 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.002618 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3956,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":19128,"spinlock_wait_cycles":723950848,"thread_start_us":359783,"threads_started":1,"wal-append.queue_time_us":713345}
I20260709 12:25:30.220486 31141 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:30.209975  2824 consensus_queue.cc:1243] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:30.193109 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.193934 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76399531471a4790a56d52a6ea052864"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:25:30.223275 31060 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:30.223382 31063 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:30.223723 31060 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.199527 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.224478 31144 raft_consensus.cc:2368] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:30.200111 31068 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:30.202137 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.197608 30990 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:30.207643 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:30.212170 31061 pending_rounds.cc:85] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 7
I20260709 12:25:30.226985 31061 pending_rounds.cc:107] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:25:30.227257 31061 raft_consensus.cc:2889] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.215551 31143 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:25:30.218103 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 83225.9us
W20260709 12:25:30.106949 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 65308.3us
I20260709 12:25:30.234028 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:30.221537 31136 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 12:25:30.179848  2983 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:30.195917 30989 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:30.223842 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 987) took 1034 ms. Trace:
I20260709 12:25:30.236070 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:30.235366 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.250962 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:30.251698 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.190407  2942 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 9. Current term is 10.
I20260709 12:25:30.201494 31058 rpcz_store.cc:276] 0709 12:25:28.879639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.881422 (+  1783us) service_pool.cc:224] Handling call
0709 12:25:28.917967 (+ 36545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.918200 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:29.128891 (+210691us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.801000 (+672109us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b180009cfa0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:25:29.981080 (+180080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:29.981133 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:29.981184 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.982147 (+   963us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.982288 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16811,"prepare.queue_time_us":180575,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":102,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":13209472,"thread_start_us":309500,"threads_started":2,"wal-append.queue_time_us":850561}
I20260709 12:25:30.251058 31692 rpcz_store.cc:276] 0709 12:25:29.189790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.189906 (+   116us) service_pool.cc:224] Handling call
0709 12:25:29.190272 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.203521 (+ 13249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.203552 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.205485 (+  1933us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.009490 (+804005us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b1800031540. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:30.157496 (+148006us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:30.157534 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.157596 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.223584 (+ 65988us) raft_consensus.cc:1685] finished
0709 12:25:30.223606 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.223791 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12874,"raft.queue_time_us":140978,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6954,"spinlock_wait_cycles":22687616,"thread_start_us":139044,"threads_started":1,"wal-append.queue_time_us":857191}
I20260709 12:25:30.254801 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.255005 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:30.256728 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:30.261492 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:30.216547  3159 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:30.265252 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:30.238731 31694 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:30.220343  2973 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:30.268342 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.269065 31692 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:30.272176 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.272589 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:30.276186 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.258150  3206 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:25:30.276835 31691 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:30.277606  2878 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.277778  3159 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:30.278043  2878 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:30.260890  3149 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:25:30.278995  2878 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.279094 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.280303 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.280937 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.281610 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2076) took 1089 ms. Trace:
I20260709 12:25:30.281980 31138 rpcz_store.cc:276] 0709 12:25:29.192123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.198119 (+  5996us) service_pool.cc:224] Handling call
0709 12:25:29.198415 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.054018 (+855603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.054063 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.054603 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.178338 (+123735us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:30.178373 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.178433 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.281387 (+102954us) raft_consensus.cc:1685] finished
0709 12:25:30.281413 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.281553 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":855216,"raft.queue_time_us":858754,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":123318,"threads_started":1,"wal-append.queue_time_us":114270}
I20260709 12:25:30.285602  3138 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:30.259305  3121 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:30.215580  3139 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:25:30.278380  3135 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:30.286849  3135 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:30.286695  3133 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:30.220494  3124 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:25:30.287303  3135 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.287352  3133 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.287937  2973 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:30.288384  2973 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.294950 31135 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:30.294891  3128 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.296721 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.296495  3123 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.297226  3128 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.297854 31689 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:30.298173 31142 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:30.254040  3115 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:25:30.276206  3122 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:30.299679 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:30.250839  3089 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:25:30.300987  3089 consensus_queue.cc:1243] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:30.301811  3121 raft_consensus.cc:2764] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:30.278815  3159 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:30.288393  3133 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.303009  3159 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.303536  3133 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.300227  3122 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:30.259536  3209 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.308096  3209 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:30.308629  3209 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.311470  3135 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:30.315539 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 60613.9us
I20260709 12:25:30.312791  2983 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.314585  3123 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.314987  3137 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:30.319036 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:30.321437 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:30.346761  2878 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.349460  2878 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.322469  3115 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:25:30.350971  3209 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.326982  3207 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:30.315864  3202 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:30.354489  3207 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:30.360231  3207 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:30.369309  3209 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.397434 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:30.164613 30884 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 3 to 5, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 5 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:30.154616 30886 rpcz_store.cc:269] 0709 12:25:00.636298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.684869 (+ 48571us) service_pool.cc:224] Handling call
0709 12:25:29.997259 (+29312390us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:22.879152 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:22.879783 (+   631us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:22.879839 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:25:22.879858 (+    19us) tablet.cc:662] Decoding operations
0709 12:25:23.113768 (+233910us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:23.134615 (+ 20847us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:23.136631 (+  2016us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:25:23.157430 (+ 20799us) tablet.cc:701] Row locks acquired
0709 12:25:23.157476 (+    46us) write_op.cc:260] PREPARE: finished
0709 12:25:23.157695 (+   219us) write_op.cc:270] Start()
0709 12:25:23.157846 (+   151us) write_op.cc:276] Timestamp: P: 1783599923157599 usec, L: 0
0709 12:25:23.157873 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:25:23.158706 (+   833us) log.cc:844] Serialized 4350 byte log entry
0709 12:25:24.091021 (+932315us) op_driver.cc:464] REPLICATION: finished
0709 12:25:24.356211 (+265190us) write_op.cc:301] APPLY: starting
0709 12:25:24.356320 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:24.584539 (+228219us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:24.584685 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:24.813829 (+229144us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:24.817784 (+  3955us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 12:25:24.832914 (+ 15130us) write_op.cc:312] APPLY: finished
0709 12:25:24.889033 (+ 56119us) log.cc:844] Serialized 105 byte log entry
0709 12:25:24.898935 (+  9902us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:24.899092 (+   157us) write_op.cc:454] Released schema lock
0709 12:25:24.899159 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:24.899559 (+   400us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:24.899587 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":262096,"reported_tablets":47,"spinlock_wait_cycles":7936,"child_traces":[["op",{"apply.queue_time_us":259370,"apply.run_cpu_time_us":23015,"apply.run_wall_time_us":543325,"cfile_cache_hit":42,"cfile_cache_hit_bytes":43911,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":72625,"lbm_reads_10-100_ms":3,"lbm_reads_lt_1ms":9,"num_ops":11,"prepare.queue_time_us":573320,"prepare.run_cpu_time_us":81010,"prepare.run_wall_time_us":884550,"raft.queue_time_us":118870,"raft.run_cpu_time_us":20051,"raft.run_wall_time_us":201127,"replication_time_us":932943,"spinlock_wait_cycles":31616,"thread_start_us":1143077,"threads_started":4,"wal-append.queue_time_us":644601,"wal-append.run_cpu_time_us":55411,"wal-append.run_wall_time_us":2109892}]]}
I20260709 12:25:30.416083 30884 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 7 to 10, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:30.410068 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:30.419838 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:30.418926 30884 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:30.424077 30884 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:30.438709 30989 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:30.441602 30984 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:30.443799 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:30.444257 31060 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:30.445108 31063 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.445283 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:30.447182 31060 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.448585 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:30.449437 31060 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.449872 31065 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:30.451609 31061 pending_rounds.cc:85] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 3
I20260709 12:25:30.452008 31061 pending_rounds.cc:107] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 12:25:30.452474 31061 raft_consensus.cc:2889] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.455673 30990 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:30.459780 31059 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:30.461326 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:30.463627 31068 raft_consensus.cc:1217] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[8.5-12.7]   Dedup: 8.5->[10.6-12.7]
I20260709 12:25:30.464195 31068 pending_rounds.cc:85] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 5
I20260709 12:25:30.464462 31068 pending_rounds.cc:107] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:25:30.464705 31068 raft_consensus.cc:2889] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:30.474167 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.481873 31060 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:30.484731  2983 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.485991  2986 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:30.492595  2973 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:30.501204  2973 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:30.501478  3232 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:25:30.495555  3135 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:30.511224 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:30.511711 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 190243us
I20260709 12:25:30.512432 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.512935  3206 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:30.516249 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.517011 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.518240 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.478282 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.524540 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.526141 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.528947 31063 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:30.532321 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.532732 31062 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:30.534710 31141 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.534965 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.537510 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.539366 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:30.540408 31695 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:30.532366 31135 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:30.542510 31142 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:30.546723 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.547179 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.547390 31135 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.548208 31142 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [8.5-11.6]
I20260709 12:25:30.548447 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.549855 31142 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:25:30.550015 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.543063 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:30.560725 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:30.565605 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.571302 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.575515 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.578513 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.581313 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.580710 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.587178 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:30.578326 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:30.589747 31141 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:30.594595 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:30.600600  3136 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:30.602290  3148 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:30.602782 31140 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:30.612985  3207 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.615758  3207 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:30.618075 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.618255  3088 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:30.622706  3128 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:30.622975 31144 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:30.623605  3128 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:30.627944 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.632484  3128 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.633023 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:30.634591 31694 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:30.638401 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.639678  3088 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:30.641193  3088 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:30.639730 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 43683.8us
I20260709 12:25:30.643222 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:30.643337 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:30.644065 31695 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:30.645596 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:30.646556  2973 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:30.647203  2973 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:30.648476  2973 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.650552  2915 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:30.652189  3089 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.652910  3089 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.654399 31067 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.602797  3148 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:30.659773  3148 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:30.660912  3148 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.652185  2649 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:30.662456  2649 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.663651  2649 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.666545 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:30.668740 31689 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:30.668900 31691 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:30.669098  2649 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.669344 31691 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.670382 31691 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.671881 31691 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.673589 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:30.674710 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.675419 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.675823 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:30.678406 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 217499us
I20260709 12:25:30.679191 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:30.683415 31142 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:30.686348 31135 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:30.687407  3181 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:30.709276  3181 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:30.709676  3181 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.692368 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.621863  3135 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:25:30.710625 31139 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:30.702646 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.703367 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.703431 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.703737 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.704432 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.705354 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.706260 31063 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:30.707242 31689 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:30.708770 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:30.707849 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.706292 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.719815 30986 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:30.720366 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.744887 31694 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:30.767973 30985 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:30.769487 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45212.8us
I20260709 12:25:30.771942 30984 raft_consensus.cc:2393] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 7: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:30.765974 30987 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:30.777297 31067 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:30.758380  3207 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:25:30.796062  3207 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:30.796634  3207 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.798894 30983 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:30.798023  3207 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.803504  3089 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:30.805544 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:30.784859 31692 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:30.786325 31065 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:30.787940  3209 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:30.791982  3201 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 12:25:30.793406 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 114330us
I20260709 12:25:30.807945 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.812426 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:30.820015 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.821149 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.820865  3209 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:30.823925  3209 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:30.826462 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:30.829416 31691 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
W20260709 12:25:30.829418 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 55263us
I20260709 12:25:30.835225 30989 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.786348  2824 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:30.839049 30958 leader_election.cc:400] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:30.840576 30958 leader_election.cc:403] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:30.841064 31695 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:30.834900  3089 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:30.847282 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.849552 30988 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:30.851509 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:30.857486 30990 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:30.855518  3209 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.849134  3135 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:30.887696 30986 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: [8.8-11.10]
I20260709 12:25:30.870788  3209 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:30.859323  2824 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:30.901379  2824 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:30.902848  3236 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:30.904654  2989 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:25:30.911755 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.912130 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.912878 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.919404 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:30.921402  3209 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.921779 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:30.926460  3206 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:30.927008  3206 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.927977  3206 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.938782 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:30.938885 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:30.939119 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:30.940491 31068 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:30.942106  3202 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:30.942818  3202 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:30.943830  3202 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:30.945272 31064 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:25:30.945632 31064 pending_rounds.cc:85] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 8
I20260709 12:25:30.945871 31064 pending_rounds.cc:107] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:25:30.945994 30984 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: [11.8-14.10]
I20260709 12:25:30.946125 31064 raft_consensus.cc:2889] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:30.947247 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:30.947816 30992 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:30.948170 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 119950us
I20260709 12:25:30.951514 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:30.952422 30983 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:30.955579  3122 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:25:30.956429  3122 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:30.956859  3122 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:30.959934  3149 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:30.960367  3149 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:30.960770  3149 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:30.965801  2878 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:30.974020 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 137181us
I20260709 12:25:30.978809 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:30.997011  3209 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.000645 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.002957 30984 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:31.004469  3134 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:31.005659  3207 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:31.036083 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 59847.8us
I20260709 12:25:31.030368  2986 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:31.021929  3146 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:31.052718  3146 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:30.948431  3135 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.053771  3146 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.055030  3146 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:31.055687  3135 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.021389  2970 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:31.056538  2970 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.058734  2970 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:31.059015 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.060334 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.060794 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:31.060838 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:31.061609 31139 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:31.061683 31142 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:31.062045 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.064321  3148 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:31.065936 31063 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:31.066284 30987 raft_consensus.cc:2410] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:30.957939  3122 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.068701 31140 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:31.069308  3122 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.064109  2986 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:31.069924  2986 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.068429 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:31.069828  3115 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:31.070410  2986 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.055528  3149 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.062455  3160 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:31.075224  3209 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:31.076591  3149 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.077545 31143 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:31.079510  2973 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:31.080423 31142 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:31.051290 30992 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
W20260709 12:25:31.080293 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.083537 30989 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
W20260709 12:25:31.087374 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.087045  2824 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:31.088294  3181 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.090543  3181 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:31.092456  2915 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.092981  2915 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.093400  3134 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:31.094368  3201 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:31.094810  3201 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:31.095177  3201 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:31.099929  3135 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:31.101016  3135 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:31.101847  3159 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:31.103108 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.106361 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.112241 30985 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:25:31.113476 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 63731.2us
I20260709 12:25:31.102288  3159 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:31.142932 31139 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:31.144634 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.145529 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:31.147919 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:31.149065 31692 raft_consensus.cc:2410] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:31.111868  2824 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 12:25:31.155516 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.156183 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.124647  2986 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.160172 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:31.132141  3181 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:25:31.162288 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:31.164074 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.141520  3136 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.176060 31068 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:31.177215 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:31.178755 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:31.124676  3201 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.182838 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:31.176558  2915 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:31.178884  3149 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:31.188911 31692 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:31.179334  3146 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:31.220088 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 78333.2us
I20260709 12:25:31.180738  3136 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.233994 31691 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:31.210573  3089 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:31.251153 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:31.255856 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 288016us
I20260709 12:25:31.230342  3088 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.232969  3146 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:31.262181 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.242906  3207 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.267716 31692 raft_consensus.cc:2410] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:31.254529  2973 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.272643 31695 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:31.276863 31691 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:31.277873 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:31.220795  3159 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.281975  3187 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.275091  3207 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.282151 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:31.286831  3089 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.288223  3121 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:31.291083  3160 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:31.296867  3136 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:31.279562  2979 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:31.306669 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.301468 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:31.270795  3135 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:31.350837 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:31.269943  3088 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.350167  2986 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:31.352846 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 357338us
I20260709 12:25:31.357547 31690 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:31.363350 31691 raft_consensus.cc:2368] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:31.337635  3160 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:31.376091 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:31.280294  2973 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.378762 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:31.379365 30992 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:31.383767 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 151997us
I20260709 12:25:31.390394 31692 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:25:31.373894  3122 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:31.392568 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:31.388607 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.378428  3159 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:31.404291 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 323454us
I20260709 12:25:31.405552 30987 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:31.408218 31695 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:25:31.411394 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 43537.3us
I20260709 12:25:31.386840  3201 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.387399  3207 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:31.352088  3187 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.424539 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.392805  3122 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:31.427736 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:31.437089 31695 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:25:31.441118 31058 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:31.440943  3122 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.442735  3122 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.445360 31058 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:31.446473 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:31.448463  3202 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:31.390802  2979 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.455646 31690 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:31.430742  3121 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:31.388933  3146 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.446748 31058 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:31.447547  3206 raft_consensus.cc:2764] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:25:31.480101 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 91611.1us
W20260709 12:25:31.516113 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.521106  3088 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:31.522043  3251 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:25:31.486093  3250 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:31.522961 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:31.544759 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 95452.6us
I20260709 12:25:31.491022 31058 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:31.571406  3260 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:25:31.574355 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48730.7us
I20260709 12:25:31.574618  3250 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:31.578660  3258 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:25:31.580597  3121 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:31.580716  3262 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:31.581257  3121 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.575281 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.543493  3148 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:31.598479  3121 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.483340  3160 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.601171 31058 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.602023  3278 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.602607  3278 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.603678 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.545055  3206 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:25:31.606024  3134 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:31.606802  3134 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:25:31.622885 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.623688 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59429.1us
I20260709 12:25:31.624024 30988 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:31.624822 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 199726us
I20260709 12:25:31.624895 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:31.625710 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:31.626135 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:31.627485 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:31.628257 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.629135  3089 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:31.634438  3089 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:31.639602 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.642022 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:31.649387 30992 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:31.615553  3122 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:31.665416 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:31.592335  3148 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.666253 31061 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:31.577620  3250 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:31.647470  3089 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:31.605047  3278 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
W20260709 12:25:31.682744 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55359.5us
I20260709 12:25:31.664525  3160 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:31.686425  3149 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:31.664873  3122 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:31.688439 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:31.615152  3135 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:25:31.672184  3086 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:31.661981  3128 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:25:31.681970  3089 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:31.681324  3187 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:31.694839 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:31.699260 31060 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:31.673341  3181 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:25:31.693638  3014 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:31.695561 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.713304 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.713716 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.688066  2649 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:31.690284  3278 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:31.709450 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.685169  2915 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:25:31.698612  2979 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.722522 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.772763 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:31.744613 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:31.774744 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 132817us
I20260709 12:25:31.725855  3149 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:31.779649 31060 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:31.725680  3255 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:31.782902  3268 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 12:25:31.784184 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61769.8us
I20260709 12:25:31.784577 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:31.784998  3287 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:31.784619  2649 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:31.798825  3122 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.799021  2649 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.800237  3122 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.801007  3148 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.776607  3283 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:31.803663  3283 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.804023  3283 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:31.790956  3287 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.777390  3128 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:31.807452 31661 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:31.814725 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 177567us
I20260709 12:25:31.805699  3089 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.780856  3149 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:31.826108  3115 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:31.826606  3115 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.826958  3115 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.831362  3288 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.831897  3288 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.810424  3287 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:31.834013  3288 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:31.852277  3272 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:25:31.791034  3278 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:31.855458  3292 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:31.855906  3278 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.856551  3292 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:31.856900  3292 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:31.860396 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 75923.1us
I20260709 12:25:31.861478 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:31.862914  3266 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:31.863703  3276 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:25:31.865000  3277 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:25:31.865218  3149 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.866284  3149 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.866662 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:31.818387 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:31.867790 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49473.3us
W20260709 12:25:31.868434 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 462579us
I20260709 12:25:31.869071 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.869701  3285 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:31.869846 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:31.870179  3285 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.870576 31060 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:31.870894  3201 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:31.871455  3201 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:31.871948 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.870615  3285 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:31.872682 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.875559 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.875885 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.876075  3128 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.877027  3265 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:31.850028  3253 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:25:31.877592  3265 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:31.878326  3128 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.878531  3265 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.880838 31058 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:31.822122 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:31.882118  2973 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.882416  2860 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:31.882939  2860 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.883361  2860 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:31.888818 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:31.889494 30989 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:31.890296 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:31.895354  3279 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:31.891551  2970 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:31.895740  3279 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:31.896000  2970 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:31.896657  3279 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.896981  2970 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.872323  3201 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.897887  3279 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:31.883467  3135 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:25:31.876528 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.914659 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.869879 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.923110  2989 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.923700  2989 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:31.925305 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:31.925635  2989 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:31.926590 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 56773.7us
I20260709 12:25:31.920935  3181 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:31.927316 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:31.927733 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:31.928553  3136 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:31.929083  3136 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:31.929859 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:31.914379  3135 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:31.929986  3136 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.931399  3135 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.932410  3135 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:31.934695 31135 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:31.935277 30989 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:31.934027  3181 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.936523 31142 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:31.936275  3181 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.937743 31143 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:31.939033 31144 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:31.940075  3181 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:31.940622  2878 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:31.941143  2878 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:31.942093  2878 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.942646  3181 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:31.947577 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:31.822659  3255 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:25:31.972378 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.845672  3287 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:31.898092  2970 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.859314  3292 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:31.984746 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.992620 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:31.994966 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:31.999063 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 46611.1us
I20260709 12:25:32.001883 31064 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:31.999375 30988 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:32.003353 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:32.000677 31058 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:32.007691 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.007946 31135 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.008749 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.011412 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:32.022483 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:32.023330 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:32.025199 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.028906 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:32.029268 31139 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:32.030100 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:32.031648  3134 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:32.032159  3134 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:32.032572  3291 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:32.033138  3134 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.035372 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:32.037473  3266 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:32.037936  3266 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.038378  3266 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.038863  3259 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:32.040390  3261 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.040946  3261 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.043280 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:32.043817 31695 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.044700 31691 raft_consensus.cc:2368] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
W20260709 12:25:32.044577 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 178072us
I20260709 12:25:32.045619 31692 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:32.008868 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.045148 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:32.049847 31143 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:32.050815 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:32.058683 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.059523 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.060988 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:32.052676  3276 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:32.048964  3292 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:32.062125 30989 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
W20260709 12:25:32.062444 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.063987 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:32.035311  3128 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:25:31.978266  3283 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.068646 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:32.070281 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.065625 30985 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:32.084482  3261 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:32.084807 31063 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:32.085072  3281 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:25:32.063810  3181 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:32.093021 31695 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:32.099408 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.101491 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.073365  3288 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:32.073030  3121 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:32.070766  3207 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:25:32.095744  3181 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:32.124467 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56696.4us
I20260709 12:25:32.127672 31143 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:32.105726  3261 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:32.124349  3159 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.704s	user 0.004s	sys 0.009s
W20260709 12:25:32.153192 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.127521  3181 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:32.170305  3290 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.005s
W20260709 12:25:32.176796 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 117535us
I20260709 12:25:32.179262 31135 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:32.177930 30988 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: [8.8-11.10]
I20260709 12:25:32.161417  3261 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:32.125379  3121 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.187544 31139 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:32.187592 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.192682 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:32.193385 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:32.193411 31692 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:32.193941 30988 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:32.140530  3282 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 12:25:32.178496  3014 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:32.177584  3296 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.180917  3086 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.214536 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:32.217075  3283 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:32.183354  3159 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:32.219897 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:32.220767 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.239027 31144 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:32.240800 31140 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:25:32.239259 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:32.166405  3187 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:25:32.247471 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:32.176626  3291 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:32.246587 31140 pending_rounds.cc:85] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 9
I20260709 12:25:32.281769 31140 pending_rounds.cc:107] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 11.10
W20260709 12:25:32.285432 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.282133 31140 raft_consensus.cc:2889] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:32.282968 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:32.221328  3287 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:32.293350 31063 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:32.291764 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48976.6us
I20260709 12:25:32.294801 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:32.184243  3115 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.301028 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:32.303750 31144 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:32.306629  3302 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 12:25:32.306843  3297 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 12:25:32.308877  3303 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:25:32.309230  3283 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.309829  3283 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.310171  2973 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.310883  2973 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.313169 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.315692 31139 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
W20260709 12:25:32.317781 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.325863 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:32.243201  3086 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.303315  3135 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:32.313015  3265 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:32.314891  3283 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:32.249346  3296 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.367905 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.296002  3159 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.376260 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.377378 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:32.379307 31144 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:32.355890  3294 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:32.394922 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.398685 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.366264  3265 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:32.368040  3250 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.412910 30992 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.414422 30987 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:32.414422 31063 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:32.418273 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:32.421633 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.412626  3265 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:32.436844  3304 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:25:32.422483  3256 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:32.417834  3294 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.438174 31697 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:32.446092 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.452629 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:32.456159 31693 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:32.424564 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:32.446193 30992 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:32.456816 31144 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.459092 30988 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:32.407058  3134 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.474502  3134 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.461746 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:32.476643 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.369951  2878 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:25:32.463733 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:32.464043 30992 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:25:32.440759  3296 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:32.479447 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 184614us
I20260709 12:25:32.448047  3294 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:32.459954  3250 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.442647  3313 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:32.466465 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:32.466728  3115 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:25:32.439316 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:32.374755  3135 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:32.464407 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 80857us
I20260709 12:25:32.468747 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:32.445807  3279 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.008s
I20260709 12:25:32.531397 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:32.531975 30988 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:32.443532  3298 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:25:32.471719 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:32.440886  3121 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:32.439594  3149 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:32.438779 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:32.382459  3261 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.475857  3159 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:32.476835 30986 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:25:32.506910 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.480639 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:32.512401 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:32.516683 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.513906 31696 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:32.514678 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.534893 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:32.535625 31143 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:32.407423  3291 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.558915 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.539542 31693 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:32.560039 30987 raft_consensus.cc:1217] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Deduplicated request from leader. Original: 6.4->[8.5-11.6]   Dedup: 11.6->[]
I20260709 12:25:32.560530 30987 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [8.5-11.6]
I20260709 12:25:32.560668 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.561235 30989 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:32.561535 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:32.562688  2860 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.003s
I20260709 12:25:32.563087  2860 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.564124 31690 raft_consensus.cc:2368] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
I20260709 12:25:32.565034  2860 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:32.568569 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.569185 31140 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:32.570001 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.578538  2973 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:32.579408 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.580367 31689 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:32.579413 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:32.581648 31140 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:32.559602 31144 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:32.544919 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.549028  3136 raft_consensus.cc:2764] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:25:32.549289  3266 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.001s
I20260709 12:25:32.549868  3086 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:32.550344  3249 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:32.552011  3135 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:32.595959 31695 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:32.554715  3294 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.555670 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.556447 30990 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:32.556998 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.557768 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:32.467727  2989 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:25:32.601145 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.597640 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.601806 30990 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
W20260709 12:25:32.585919 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 338502us
W20260709 12:25:32.586859 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.587270 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 70498.5us
I20260709 12:25:32.594075 30988 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:32.595753 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.552551  3121 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:32.598305 30983 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.630318 30986 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:32.632222 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.632835 31063 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:32.633183 30988 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:32.634114 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:32.636796 30988 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.593627  3134 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:32.647557 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.648981 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.651917 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.654314 30990 raft_consensus.cc:2368] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:32.648173 31059 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
W20260709 12:25:32.656792 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 77701.4us
I20260709 12:25:32.602747  3249 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:32.647145 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:32.658115 30984 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:32.658447 30986 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:32.660493  3250 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:32.628844  2649 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.594506  3266 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:32.675616 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 211939us
I20260709 12:25:32.625679  3294 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.664176  3089 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:32.664631 30988 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:32.683516 30987 raft_consensus.cc:2368] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12.
W20260709 12:25:32.682180 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 201599us
W20260709 12:25:32.688987 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.647378 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:32.685269 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:32.706439 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:32.677681  3279 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.736919 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:32.740365 31059 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.740536 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:32.741183 31062 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:25:32.736286 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 106822us
I20260709 12:25:32.736591 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:32.739719 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:32.742995 31063 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:32.717087 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 113449us
I20260709 12:25:32.742995 31143 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:32.752166 30983 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
W20260709 12:25:32.753875 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 68708.5us
I20260709 12:25:32.757761 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:32.683907  2860 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:32.758839 31061 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:25:32.754196 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 79460.1us
I20260709 12:25:32.755343 31063 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
W20260709 12:25:32.755460 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 71192.3us
I20260709 12:25:32.762785 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:32.673355  2649 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.765641 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:32.763253 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:32.768637 31137 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:25:32.763727 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 74865.8us
I20260709 12:25:32.774771  3287 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:32.775251  3187 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:32.776365  3202 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:32.776489  3285 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.886s	user 0.011s	sys 0.002s
I20260709 12:25:32.736023  2878 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:32.797806 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:32.764645  3313 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:32.778539 31139 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:32.778853 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:32.737035  2915 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.790439 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:32.802709 31143 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:32.814074 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.754760  3279 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.787154  3287 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.846462  3332 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:32.839406  3314 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 12:25:32.804525  3285 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.798205  2878 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:32.800735  3320 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:32.799412  3201 raft_consensus.cc:2764] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
W20260709 12:25:32.854503 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 66815.1us
W20260709 12:25:32.856005 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 65607.7us
I20260709 12:25:32.872402 31059 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:32.780663  2860 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:32.861945  3122 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.061s	user 0.013s	sys 0.001s
I20260709 12:25:32.885996  3121 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:32.797730  3202 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:32.886571 31689 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:32.886848 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:32.889797 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 99441.4us
I20260709 12:25:32.890103 31696 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:32.876786  3259 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:32.851198  3333 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:32.918115  2986 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:32.890579 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.564s (SENT) [suppressed 13 similar messages]
I20260709 12:25:32.892263  3294 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:32.893704  2649 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:32.811405  3278 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:25:32.949648 31695 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:32.854946  3285 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:32.896066  3279 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:32.838573  2915 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:32.897310  3181 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.002s
I20260709 12:25:32.854679  2878 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.873904  3313 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:32.898968  2860 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:32.857400  3123 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:32.853219  3332 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:32.861810  3287 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:32.876313  3317 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 12:25:32.886183 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.906482 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.966779 31059 raft_consensus.cc:2410] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:32.906940  3121 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:32.907884 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:32.909814 31109 leader_election.cc:400] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:32.784044  3249 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:32.917524  3322 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.369s	user 0.000s	sys 0.003s
I20260709 12:25:32.919356 31144 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:32.925472 31140 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:32.913906  3266 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:32.959965 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.927145  3122 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:32.921404  3259 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:32.926045  2986 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:32.956847  3201 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.992587 31141 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:32.993605 30991 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:32.994009 31691 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:32.996207  3259 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:32.996367 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 86683.8us
I20260709 12:25:32.997107  2986 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.054947  2986 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.068377 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:32.996847  3313 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.069440  3122 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.070123  3122 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:32.998553  2915 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:33.001920 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:33.075027 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 275677us
I20260709 12:25:33.075402 31060 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.003998  3202 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:32.969514  3121 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:33.009011 31140 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:33.009713 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:33.010838  3292 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.941s	user 0.012s	sys 0.000s
I20260709 12:25:33.011230 31144 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:32.995078  3123 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:33.018903  3332 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:33.020304  3265 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.001s
W20260709 12:25:33.027377 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 119578us
I20260709 12:25:33.027846  3255 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:33.027875  3338 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.078040 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:32.965513  3187 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:33.078890 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:33.073910  3346 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.005s
I20260709 12:25:33.097615 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:33.102188 31142 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
W20260709 12:25:33.103793 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.105329 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:32.969539  3333 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:33.106150 31059 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:33.081578 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 268053us
I20260709 12:25:33.107452 31143 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:33.108223 31062 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
W20260709 12:25:33.110754 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 101138us
I20260709 12:25:33.124792 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:32.964761  3181 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.117825 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:33.129782 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:33.106068  3288 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:33.137801 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.139382 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:33.141920 30988 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.144295 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:33.145320 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.146453 30983 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:33.131072  3292 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:33.154973 31140 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:33.151794 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.154008 30984 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:33.159771 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:33.161196 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:33.163893 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.079504  3201 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.135619  3291 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.106515  3265 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.176321 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:33.144960  3135 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:33.186112 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:33.152510  2970 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.148172  3255 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:33.187038 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 296470us
I20260709 12:25:33.148337  3206 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:33.135716  3202 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:25:33.153438  3292 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.197889 31058 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:33.172240  3122 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:33.202275 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.204103 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:33.206171 31060 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
W20260709 12:25:33.213029 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 107333us
I20260709 12:25:33.182685  3134 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:33.217372 31059 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:33.186796  3135 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.222602 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:33.118311  3338 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.218719 30987 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:33.224905 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:33.219226 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:33.219908 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:33.196450  2860 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:33.217769 30984 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:33.224241 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:33.233911 31692 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:33.208108  3292 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:33.229218  2860 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.233588 31697 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:33.208330  3294 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:33.243988 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:33.184532  3291 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.247172 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:33.248432 30988 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:33.249718 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.221132  3312 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.252343 31033 leader_election.cc:400] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:33.254964 31033 leader_election.cc:403] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:33.225934  3135 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:33.255903 31693 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:33.258173 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.258404 31033 leader_election.cc:400] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
I20260709 12:25:33.197243  2970 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.238900  3255 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:33.214280  3206 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.251952  2860 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.165760  3123 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.264464 30984 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:33.183166  3121 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.257658  3332 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:33.254115  3136 raft_consensus.cc:2764] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:25:33.260630  3160 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:33.261895 30988 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:33.260007  3249 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:33.274533 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 52028.1us
I20260709 12:25:33.261063  3285 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:33.282341 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:33.283495 31139 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.265705  3294 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:33.217667  3187 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.272511  3255 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:33.293434 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.300114 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.303442 30988 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:33.242291  3333 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.315708 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:33.290189  3332 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:33.261384  3312 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.296411  3136 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.301570  3285 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:33.342959 31139 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:33.315263  3181 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.290938  3348 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:25:33.326588  3121 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:33.294695  3331 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:25:33.318076  3335 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:33.340842  3250 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:33.482272 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 55341.2us
W20260709 12:25:33.465335 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 149712us
I20260709 12:25:33.489917 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:33.387494  3278 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.345644  3136 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:33.509006 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.429275  3290 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.505692  3134 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.517721  3134 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:33.518743  3355 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:25:33.520028  3134 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:33.520459  3365 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:25:33.521811  3315 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.101s	user 0.003s	sys 0.003s
W20260709 12:25:33.505965  3357 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:25:33.504017  3249 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:33.568607 30958 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:33.591042 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.650570 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 82091.1us
I20260709 12:25:33.515254  3265 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:33.522528  3338 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.660774 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:33.673156 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:33.464830  3294 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:33.678023 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.483673  3335 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.694680  3312 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:33.695194  3291 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.696209  3335 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:33.507373  3278 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.698467  3206 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:33.698750  3278 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:33.680929  3362 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
W20260709 12:25:33.681756  3364 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.299s	user 0.000s	sys 0.000s
I20260709 12:25:33.686401  3372 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:33.690397  3363 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:25:33.508989  3332 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.693709  3201 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.660255  3089 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:33.707927  2986 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.653s	user 0.004s	sys 0.005s
I20260709 12:25:33.498745  3255 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.652324  3285 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.679491  3338 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:33.678130  3249 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:33.524217  3290 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.746264 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:33.661634  3283 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:25:33.747928  3372 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:33.766903  3259 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.700s	user 0.010s	sys 0.000s
W20260709 12:25:33.770848  3135 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
W20260709 12:25:33.774523  3287 consensus_meta.cc:287] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.001s
W20260709 12:25:33.785367 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 112366us
I20260709 12:25:33.772279  3249 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.792363 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:33.795433 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:33.804831 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:33.805371 31689 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.808241 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:33.808818 31697 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:33.810949 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:33.806082  3089 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.809544  3250 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:25:33.813294  3250 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:33.813735  3250 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:33.814798  3250 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.816511  3250 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:33.819604  3359 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.427s	user 0.001s	sys 0.000s
I20260709 12:25:33.824543 30986 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:33.806854 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:33.782130  3135 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.826452  3283 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.827607 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:33.828075 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 820) took 1204 ms. Trace:
I20260709 12:25:33.828233  3135 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:33.828486 30983 rpcz_store.cc:276] 0709 12:25:32.623228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.757838 (+134610us) service_pool.cc:224] Handling call
0709 12:25:32.758400 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.146945 (+388545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.827859 (+680914us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b540011fdd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:33.828001 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":891799808}
I20260709 12:25:33.829806  3283 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.831868 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:33.832758  3338 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:33.845141  2970 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:33.846082 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2110) took 1421 ms. Trace:
I20260709 12:25:33.846575 31137 rpcz_store.cc:276] 0709 12:25:32.424592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.790673 (+366081us) service_pool.cc:224] Handling call
0709 12:25:32.790992 (+   319us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:32.847731 (+ 56739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.847932 (+   201us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:32.848635 (+   703us) log.cc:844] Serialized 78 byte log entry
0709 12:25:33.219462 (+370827us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:33.233951 (+ 14489us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.234376 (+   425us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.234669 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.312728 (+ 78059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.312898 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.719177 (+406279us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b180040f180. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a7522 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:33.838769 (+119592us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b180040f180. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:33.841894 (+  3125us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b180040f180. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:33.845798 (+  3904us) raft_consensus.cc:1685] finished
0709 12:25:33.845827 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.845986 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15091,"prepare.queue_time_us":115810,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":7550,"raft.queue_time_us":72327,"raft.run_cpu_time_us":483,"raft.run_wall_time_us":17532,"spinlock_wait_cycles":27282304,"thread_start_us":428533,"threads_started":2,"wal-append.queue_time_us":464551}
I20260709 12:25:33.817220 31689 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:33.781299  3259 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.833864 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:33.790608  3249 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:33.862433  3338 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:33.862988  3338 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:33.864075 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.863857  3338 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.864578 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:33.865235  3288 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:33.865696  3288 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:33.866381  3250 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:33.866779  3250 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.867130  3250 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:33.869408  3292 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.615s	user 0.008s	sys 0.000s
W20260709 12:25:33.870810  3376 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:33.871515 30986 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:33.864622 30990 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:33.876658  2878 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.878768  2970 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.879241  2970 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.881250  2970 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:33.882973 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.883292 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.883855 30988 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
W20260709 12:25:33.884694 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.885174 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.885900  3336 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:33.886245  2986 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:33.886924  3336 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:33.895737 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 660) took 1011 ms. Trace:
I20260709 12:25:33.902128  3201 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:33.901934  3329 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:33.902606  3201 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:33.903512  3201 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.904614  3201 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.906454  2989 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:33.909076  3288 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.911454  3288 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:33.912170 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.912465 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.912745 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.913048 30987 raft_consensus.cc:2376] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14. Re-sending same reply.
I20260709 12:25:33.913293 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:33.913360 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.913529 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:33.913975 30990 raft_consensus.cc:2376] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12. Re-sending same reply.
I20260709 12:25:33.914301 31140 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:33.920603 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:33.921110 30990 raft_consensus.cc:2376] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14. Re-sending same reply.
I20260709 12:25:33.921139 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.896214 31695 rpcz_store.cc:276] 0709 12:25:32.883978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.887187 (+  3209us) service_pool.cc:224] Handling call
0709 12:25:32.887472 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.950049 (+ 62577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.895471 (+945422us) spinlock_profiling.cc:243] Waited 61.8 ms on lock 0x7b54005564d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:33.895666 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142129408}
I20260709 12:25:33.930007 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:33.931038 31139 raft_consensus.cc:2410] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:33.933053 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:33.933596 31138 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:33.934187 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:33.935978 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:33.869817  3292 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.938586 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:33.939070 30991 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:33.939107 31691 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:33.918335 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.943887  3159 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.944365  3159 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.944721  3249 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.945250  3320 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.945773  3320 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.946409  3159 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.946914  3159 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.947337  3159 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.947917  3381 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:33.948643 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:33.949538  3348 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:33.949848 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:33.901793  3181 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:25:33.947729  3320 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.950374 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:33.948462  3381 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.965934 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:33.950070 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:33.967118 31136 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:33.952741 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.968045 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:33.953711 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:33.969096 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:33.969276 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:33.956241 31140 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:33.969779 31143 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16.
I20260709 12:25:33.956971 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.950335 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:33.960871 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:33.965471  2989 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:33.950349  2986 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:33.965873 31690 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:33.788818  3287 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:33.968954 30986 raft_consensus.cc:2376] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12. Re-sending same reply.
I20260709 12:25:33.950062  3283 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.975135 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:33.991715 31033 leader_election.cc:400] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10.
I20260709 12:25:33.993018 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:33.993407 31137 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:33.949918  3159 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.994908 31033 leader_election.cc:400] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12.
I20260709 12:25:33.975176  3134 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:33.978173 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:33.950492  3331 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:25:34.005861 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:34.006808 31137 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:33.996169 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:34.022782 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 52013.2us
I20260709 12:25:33.967118  3128 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:34.033414  3128 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.034363  3259 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:34.035532  3128 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.971800  3122 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.975590  2989 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.037827  3122 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:33.968102  3121 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.039855  3121 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.040557  3122 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:34.038493  2989 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.026100  2986 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.026805  3349 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.049058  3250 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:34.034471  2860 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.757s	user 0.005s	sys 0.005s
I20260709 12:25:34.054526  2860 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.058015  2860 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.058497  3371 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:34.042006  3121 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:34.059033  3373 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:34.059095  3371 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.059659  3373 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:34.060058  3373 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:34.061350  3371 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:34.061982  3329 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:25:34.063330  3287 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.064286  3292 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:33.991154  3265 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:25:34.065659  3265 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:34.065995  3265 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:34.066274  3336 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:25:34.066915  3265 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.068145  3265 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.022735  3136 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.005721  3312 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:34.070812  3134 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:34.006102  3283 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.072180 30985 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:34.073196  3283 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.074700  3381 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.075057  3290 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.075716  3249 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:34.076403 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 52212.8us
I20260709 12:25:34.077050 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:34.025699  3377 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.077791  3377 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.078186  2989 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.078866 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:34.079799  3377 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:33.997526  3294 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.971544  3348 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.081167  3296 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:34.080670  2989 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.082171  3296 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:34.082603  3296 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.082901  3348 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.083953  3379 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:34.084352  3379 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:34.085216  3379 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.091084 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:34.091902 31141 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:34.092360 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:34.092947 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:34.093267 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2108) took 1705 ms. Trace:
I20260709 12:25:34.093667 31142 rpcz_store.cc:276] 0709 12:25:32.387662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.423324 (+ 35662us) service_pool.cc:224] Handling call
0709 12:25:34.093104 (+1669780us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b540006fed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:34.093191 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487310208}
I20260709 12:25:34.092988 31137 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:34.094004 31141 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:34.096244 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:34.098829 30992 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2255) took 1520 ms (client timeout 1671 ms). Trace:
W20260709 12:25:34.099246 30992 rpcz_store.cc:269] 0709 12:25:32.578109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.582911 (+  4802us) service_pool.cc:224] Handling call
0709 12:25:34.098643 (+1515732us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54002c1354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:34.098754 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1377361792}
I20260709 12:25:34.093282 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:34.103367 31143 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
W20260709 12:25:34.104880 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.108191 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:34.082924  3296 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.116685 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:34.117602 31693 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 8: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:34.118198 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:34.121595 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.133756 31060 raft_consensus.cc:2410] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:34.122890 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:34.135337 31068 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:34.122295 31059 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:34.123602 31138 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:34.137462 31137 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:34.119292 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:34.156437 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 125272us
I20260709 12:25:34.156536 31690 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:34.156179 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:34.159507 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:34.159690 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.161293 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.167976 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.168679 30989 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:34.169679 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.169960 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.171815 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.173893 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.174674 31062 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:34.177825 30986 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:34.178508 31061 raft_consensus.cc:2410] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:34.170745 30990 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:34.178793 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.170486 30984 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:34.179749 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.180053 30984 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:25:34.181254 31062 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:25:34.181798 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 57493.7us
I20260709 12:25:34.182731 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.181272 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:34.222818 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:34.223112 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:34.223511 30992 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:34.223634 30984 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.224727 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:34.225764 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44572.1us
I20260709 12:25:34.226549 30958 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16.
I20260709 12:25:34.226845 30958 leader_election.cc:403] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:25:34.228243 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 72164.8us
I20260709 12:25:34.235093 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:34.241837 31143 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:34.239174 30990 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:34.239049 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:34.073196  3372 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.177763  3379 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:34.155936  3349 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:34.239383 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:34.273070 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.349757 30988 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:34.232226 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.327097 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:34.333796 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:34.333473 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:34.333112 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:34.351419 31690 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:34.351854 31144 raft_consensus.cc:2410] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:25:34.331681 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:34.352516 30989 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:34.342377 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:34.343343 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 99621.7us
I20260709 12:25:34.239599 31138 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:34.367125 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:34.240293  3320 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.368288  3320 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.369226 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:34.188086  3283 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.386019 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.389539 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.244457 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.403808 30986 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:34.367427 30987 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:34.238996 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:34.376629 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:34.406859 31691 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:34.390606 31063 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:34.243813  3259 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:34.239213 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:34.410912 31059 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:34.408327 30992 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:34.442353 31060 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:34.443384 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:34.442080 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 99841us
I20260709 12:25:34.444738 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:34.446171 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:34.341473  3377 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.459100 31143 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:34.466195 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:34.370786  3372 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:34.461444 31065 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:34.377976  3207 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:34.478482 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 92021.9us
W20260709 12:25:34.478977 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 102441us
I20260709 12:25:34.482625 30989 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:34.483994 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:34.488893 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:34.227784  3249 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.237s
I20260709 12:25:34.239696  3261 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:34.444425  3181 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16.
I20260709 12:25:34.447976  3332 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:34.390472  3160 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:34.495060 31143 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:34.494938 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.495550  3371 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.495854 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.499540 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:34.565573 31063 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:34.499778  3323 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.501984 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:34.448696  3086 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:34.502794  3382 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:25:34.461038  2986 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:34.491984  3115 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:34.503317  3014 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:34.504218  2989 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.504251  3348 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:34.486207  3285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.012s	sys 0.019s
I20260709 12:25:34.505399  3201 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:34.493525  3259 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:25:34.413946  3349 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.464186  3122 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:34.474532  3377 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.474097  3320 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:34.573725  3291 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:34.527803 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.628643 31059 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:34.632251 31063 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:34.556272 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.558127 30984 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:34.562723 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:34.536406  3187 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.005s	sys 0.009s
I20260709 12:25:34.575022 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:34.640635 31144 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.576949 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:34.641870 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 65018.3us
I20260709 12:25:34.643263 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:34.577466 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 75568.9us
W20260709 12:25:34.571557  3333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.017s	sys 0.021s
I20260709 12:25:34.519353  3250 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:34.522495  3283 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 12:25:34.610019 31696 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.013s	sys 0.013s
W20260709 12:25:34.813036 31696 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.013s	sys 0.013s
I20260709 12:25:34.522231  3296 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.526324  3372 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.579115  3115 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.379222  2915 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
W20260709 12:25:34.667258  3333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.017s	sys 0.021s
W20260709 12:25:34.588189  3285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.012s	sys 0.019s
I20260709 12:25:34.611126  3201 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:34.627398  3356 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.000s	sys 0.058s
I20260709 12:25:34.607378  3312 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:34.611168  3265 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.001s
I20260709 12:25:34.607594  3348 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.666062  2986 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:34.544198 31141 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
W20260709 12:25:34.635377 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 72741.9us
W20260709 12:25:34.692521 31139 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.012s	sys 0.015s
I20260709 12:25:34.570777  3160 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.786072 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:34.851547 31139 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.012s	sys 0.015s
I20260709 12:25:34.569952  3332 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.574486  3135 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
W20260709 12:25:34.853154 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 67164us
I20260709 12:25:34.851017 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:34.785252  3255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.007s	sys 0.012s
W20260709 12:25:34.640204  3187 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.006s	sys 0.009s
I20260709 12:25:34.579306  3371 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:34.786947  3281 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.008s	sys 0.015s
I20260709 12:25:34.633111  3328 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.231s
I20260709 12:25:34.640339  3291 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:34.837325  3329 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:34.787179  3086 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.840940  3115 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.855166 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:34.889784 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:34.846236  3356 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.000s	sys 0.059s
I20260709 12:25:34.580646 31143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:34.892879 31143 raft_consensus.cc:606] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:34.893328 31143 raft_consensus.cc:614] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:34.841393  3249 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:34.605887  2989 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.813920  3292 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:25:34.845679  3312 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.844458  3159 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:34.822042  3338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.004s	sys 0.016s
W20260709 12:25:34.870004 31692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.078s	sys 0.127s
W20260709 12:25:34.916102 31692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.079s	sys 0.127s
W20260709 12:25:34.916256 30958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.002s	sys 0.014s
W20260709 12:25:34.858399  3281 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.008s	sys 0.016s
I20260709 12:25:34.874001  3291 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:34.874647  3086 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.876473  3160 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:34.858186  3014 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:34.856272  3255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.007s	sys 0.013s
I20260709 12:25:34.857303  3287 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:34.930624  3014 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:34.931025  3201 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.931576  3014 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.932829 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 662) took 1014 ms. Trace:
I20260709 12:25:34.933188 31696 rpcz_store.cc:276] 0709 12:25:33.918274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.918419 (+   145us) service_pool.cc:224] Handling call
0709 12:25:33.943972 (+ 25553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.951702 (+  7730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.951731 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.966235 (+ 14504us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.813517 (+847282us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:34.813575 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.813668 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.932637 (+118969us) raft_consensus.cc:1685] finished
0709 12:25:34.932655 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.932774 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33968,"raft.queue_time_us":16133,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":8218,"thread_start_us":846893,"threads_started":1,"wal-append.queue_time_us":960656}
I20260709 12:25:34.936017  3377 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:34.936064  3371 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:34.921772 31143 pending_rounds.cc:85] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 9
I20260709 12:25:34.854029  2986 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:34.923358  3288 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:34.939836 31143 pending_rounds.cc:107] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:25:34.940006 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:34.882166 31068 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.724s	user 0.005s	sys 0.000s
I20260709 12:25:34.940685 31064 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
W20260709 12:25:34.909942 31689 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.008s	sys 0.016s
I20260709 12:25:34.905752  2860 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:34.941655 31689 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.008s	sys 0.016s
I20260709 12:25:34.905416  3265 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:34.915287  3312 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:34.915614  3159 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.927232  2989 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.927829  3086 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:34.929127  3349 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:34.937369 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.940183 31143 raft_consensus.cc:2889] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:34.956894 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.009s	sys 0.007s
W20260709 12:25:34.957183 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.009s	sys 0.007s
W20260709 12:25:34.957464 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 940620us
I20260709 12:25:34.940807 31068 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
W20260709 12:25:34.882166  3379 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.532s	user 0.004s	sys 0.000s
W20260709 12:25:34.911763 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.939463  2986 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:34.948940 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:34.959561 31142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.006s	sys 0.022s
W20260709 12:25:34.959991 31142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.006s	sys 0.022s
I20260709 12:25:34.949018 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:34.950706 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:34.918679  3338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.004s	sys 0.021s
I20260709 12:25:34.898518  3187 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:34.899682  3123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.044s	user 0.034s	sys 0.022s
W20260709 12:25:34.920625 30958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.002s	sys 0.014s
I20260709 12:25:34.897425  3249 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:34.940452  3373 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.880s	user 0.011s	sys 0.000s
I20260709 12:25:34.963109 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:34.952739  3159 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.975807 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:34.958568 31061 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:34.957548  3332 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:34.969465 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 741703us
I20260709 12:25:34.961954  3121 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:35.028292 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:35.029719 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:34.995707  3123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.140s	user 0.034s	sys 0.023s
W20260709 12:25:35.011178 30990 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.028s
W20260709 12:25:35.035872 30990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.000s	sys 0.028s
W20260709 12:25:34.956264 31136 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.006s	sys 0.009s
I20260709 12:25:34.986503 31144 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:25:35.033699 31135 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.037s	sys 0.076s
I20260709 12:25:34.968981  3187 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:35.036944 31136 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.007s	sys 0.010s
W20260709 12:25:35.037400 31135 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.038s	sys 0.076s
I20260709 12:25:35.038189 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2125) took 1080 ms. Trace:
I20260709 12:25:34.967445  3086 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:35.039074  3332 raft_consensus.cc:2764] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:35.040498 31139 rpcz_store.cc:276] 0709 12:25:33.957908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.958084 (+   176us) service_pool.cc:224] Handling call
0709 12:25:33.958417 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.958651 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:34.851931 (+893280us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.037774 (+185843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:35.037837 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:35.037900 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.037979 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.038121 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":872953,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2395,"raft.queue_time_us":84,"thread_start_us":1051502,"threads_started":2}
I20260709 12:25:35.046348 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:35.031814  3385 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:25:35.033136  3393 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:25:35.031471  3377 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:35.036301  3338 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:35.031280  3379 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:35.051203  3135 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.055557  3394 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:35.063548  3394 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:35.064457  3394 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.068349 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 665) took 1119 ms. Trace:
W20260709 12:25:35.070240  3389 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:25:35.068914 31692 rpcz_store.cc:276] 0709 12:25:33.949330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.950956 (+  1626us) service_pool.cc:224] Handling call
0709 12:25:33.951286 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.971778 (+ 20492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.971809 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.975032 (+  3223us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.916565 (+941533us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:34.916619 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.916688 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.068036 (+151348us) raft_consensus.cc:1685] finished
0709 12:25:35.068068 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.068245 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26843,"raft.queue_time_us":37643,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":8576,"thread_start_us":920433,"threads_started":1,"wal-append.queue_time_us":1070702}
W20260709 12:25:35.044318  3388 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
I20260709 12:25:34.962121  2989 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:25:35.050658  3086 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.086185  3086 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.062247  3338 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.090034  3338 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.095793 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1013) took 1225 ms. Trace:
I20260709 12:25:35.096554  3329 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:35.097024  3329 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:35.039284 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 63562.4us
W20260709 12:25:35.049099  3386 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:25:35.033396  2649 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.096266 31689 rpcz_store.cc:276] 0709 12:25:33.870655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.919085 (+ 48430us) service_pool.cc:224] Handling call
0709 12:25:33.919363 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.919744 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.919772 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.951458 (+ 31686us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.942123 (+990665us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:34.965912 (+ 23789us) log.cc:844] Serialized 14 byte log entry
0709 12:25:34.997770 (+ 31858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.997864 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.095555 (+ 97691us) raft_consensus.cc:1685] finished
0709 12:25:35.095583 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.095729 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40392,"prepare.queue_time_us":26567,"prepare.run_wall_time_us":41,"raft.queue_time_us":51238,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":5038,"spinlock_wait_cycles":6697216,"thread_start_us":920140,"threads_started":1,"wal-append.queue_time_us":1112275}
I20260709 12:25:34.953219  3265 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.049227 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:35.084547 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.000s	sys 0.036s
I20260709 12:25:35.044276  3320 raft_consensus.cc:2764] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:35.044052  3187 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:34.873009  3354 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.422s
I20260709 12:25:34.996815  3373 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.110131  3373 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:35.110954 31062 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:35.099611 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:35.053015  3255 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:35.115676  3373 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:35.115919  3187 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:35.116901 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.000s	sys 0.036s
I20260709 12:25:35.083484  3296 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:35.119150 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:35.120469 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 71328.6us
I20260709 12:25:35.130923 31067 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:25:35.131917 31067 pending_rounds.cc:85] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 8
I20260709 12:25:35.051520  3323 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:25:35.098861  3372 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:35.132205 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 854967us
I20260709 12:25:35.132345 31067 pending_rounds.cc:107] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:25:35.128693  3335 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:35.135288 30957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.004s	sys 0.013s
W20260709 12:25:35.145040 30957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.004s	sys 0.014s
W20260709 12:25:35.136137  3397 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:25:35.137903  3373 raft_consensus.cc:2764] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:35.138883 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:35.139420  3261 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:35.140213  3089 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.140496  3265 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.121939  2649 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.141283 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:35.141450  3404 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.143939 31067 raft_consensus.cc:2889] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:35.061992  3377 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.121920  3255 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:35.062755  3135 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.150501 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 814) took 1044 ms. Trace:
I20260709 12:25:35.151042  3159 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.060880  3115 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.134055  3348 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.178066 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2291) took 1053 ms. Trace:
I20260709 12:25:35.178483 31067 rpcz_store.cc:276] 0709 12:25:34.119698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.120729 (+  1031us) service_pool.cc:224] Handling call
0709 12:25:35.129906 (+1009177us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b5800129398. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:25:35.130154 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.173180 (+ 43026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.173358 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4910336}
I20260709 12:25:35.119236  3249 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.161434 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:35.168706 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:35.166380  3373 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:35.198349  3408 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:25:35.199615 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2303) took 1226 ms. Trace:
I20260709 12:25:35.203612 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:35.206363 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 816) took 1096 ms. Trace:
I20260709 12:25:35.171792  2989 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:35.175522 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.008s	sys 0.023s
W20260709 12:25:35.218758 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.009s	sys 0.023s
W20260709 12:25:35.176079 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.043s	sys 0.075s
W20260709 12:25:35.219516 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.043s	sys 0.075s
I20260709 12:25:35.177999  3403 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:35.158155  3089 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.158540 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 809) took 1067 ms. Trace:
I20260709 12:25:35.222393 31142 rpcz_store.cc:276] 0709 12:25:34.091149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.094558 (+  3409us) service_pool.cc:224] Handling call
0709 12:25:34.094826 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.230457 (+135631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.230499 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.231047 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.960406 (+729359us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:34.960459 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.960546 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.158316 (+197770us) raft_consensus.cc:1685] finished
0709 12:25:35.158336 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.158473 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173874,"raft.queue_time_us":136108,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":107934,"spinlock_wait_cycles":25984,"thread_start_us":572813,"threads_started":1,"wal-append.queue_time_us":816583}
I20260709 12:25:35.200021 31140 rpcz_store.cc:276] 0709 12:25:33.973076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.973234 (+   158us) service_pool.cc:224] Handling call
0709 12:25:33.973609 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.997005 (+ 23396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.997050 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.382578 (+385528us) spinlock_profiling.cc:243] Waited 7.61 ms on lock 0x7b2c000e06f8. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:25:34.382926 (+   348us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.960921 (+577995us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:34.960974 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.961137 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.199284 (+238147us) raft_consensus.cc:1685] finished
0709 12:25:35.199304 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.199533 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93868,"raft.queue_time_us":6130,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":20135,"spinlock_wait_cycles":17506176,"thread_start_us":467762,"threads_started":1,"wal-append.queue_time_us":580090}
W20260709 12:25:35.171233 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 528012us
I20260709 12:25:35.224993 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:35.219805 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 633999us
I20260709 12:25:35.227607 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:35.230792 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:35.172806 31135 rpcz_store.cc:276] 0709 12:25:34.106006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.107026 (+  1020us) service_pool.cc:224] Handling call
0709 12:25:34.107318 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.231357 (+124039us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:34.231384 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.240472 (+  9088us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.041793 (+801321us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:35.041851 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.041985 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.150295 (+108310us) raft_consensus.cc:1685] finished
0709 12:25:35.150319 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.150442 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139650,"raft.queue_time_us":124229,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":59405,"thread_start_us":628539,"threads_started":1,"wal-append.queue_time_us":799597}
I20260709 12:25:35.232196 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:35.198495  3121 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:35.194623  3348 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:35.236356 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:35.200177  3249 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:35.238435 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:35.242951 31689 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:35.244064 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.245159 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.245944 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.247205 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.247783 30986 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:35.207098  3381 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:35.250167 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.175590  3404 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.251281 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.253909 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.314641 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:35.315485 31689 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:35.256417 31067 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:35.317087 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.261346 30987 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:25:35.262550 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.319171 30984 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:35.276360 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:35.253909 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:35.311959 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.312377 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.324168 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:35.324736 30991 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:35.324752  3371 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:35.326207 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 107265us
W20260709 12:25:35.327742 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 72393.9us
I20260709 12:25:35.331941 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:35.332417 30988 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:35.239346  3121 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.329421 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:35.328305 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 74463.8us
W20260709 12:25:35.197522  3250 consensus_queue.cc:1175] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce27a72886964e4287059a1d66bd5b6b" responder_term: 12 status { last_received { term: 12 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 12 index: 5 } }
I20260709 12:25:35.239473  3202 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:35.240024  3407 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:35.249532  3282 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:35.311882  3265 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:35.312877  3394 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:35.353720  3394 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:35.354306  3394 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:35.342007 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:35.228720  2989 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.362974  2989 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:35.260885  2986 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:35.239960  3296 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:25:35.251473  3335 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:35.344772 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:35.345160 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:35.327946  3281 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.324570  3292 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:35.239514  3291 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 12:25:35.323135  3380 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:35.368872  3406 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:25:35.239960  3403 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.369230  3413 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.346567  3265 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.339219  3372 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:35.399181  3415 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 12:25:35.399315  3265 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:35.399997  3418 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:25:35.401829  3404 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:35.341917  3261 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:25:35.325116  3255 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.387667  3282 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:35.389623 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 820) took 1152 ms. Trace:
I20260709 12:25:35.392961 31692 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:35.393298 30991 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:35.371109 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:35.370812  2986 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.414774 31143 rpcz_store.cc:276] 0709 12:25:34.237323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.921214 (+683891us) service_pool.cc:224] Handling call
0709 12:25:34.921564 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.957280 (+ 35716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.957309 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.957703 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.234055 (+276352us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:35.234409 (+   354us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.234715 (+   306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.234790 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.389361 (+154571us) raft_consensus.cc:1685] finished
0709 12:25:35.389389 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.389535 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6557,"prepare.queue_time_us":3184,"raft.queue_time_us":88,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":11965,"spinlock_wait_cycles":146108800,"thread_start_us":276092,"threads_started":1,"wal-append.queue_time_us":429729}
I20260709 12:25:35.427261 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:35.376300  3333 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:25:35.376664  3348 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:25:35.423041 30988 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:35.427953 31143 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:35.384007 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.443706  3265 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.445022  2986 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:35.444927  3335 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.445700  3265 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.446511  3135 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:35.447466  3282 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.445634  2986 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.448971  2986 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.450922  2986 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:35.451359 31143 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:35.452361 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.456645 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.399998  3411 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:25:35.457767 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.459054 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.398306  3380 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.463166 31694 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:35.464110  3089 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:35.464324  3380 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.468532  3407 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:35.391340  3281 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.476430  3281 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.476987  3377 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:35.477319  3403 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.435412  3426 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:35.478825  3426 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.206795 31136 rpcz_store.cc:276] 0709 12:25:34.109242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.118669 (+  9427us) service_pool.cc:224] Handling call
0709 12:25:34.118939 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.119111 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:35.040105 (+920994us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.205725 (+165620us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:35.205790 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:35.205847 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.205926 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.206063 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":750177,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":38,"raft.queue_time_us":1829,"thread_start_us":1085716,"threads_started":2}
I20260709 12:25:35.479923  3333 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:35.480858  3426 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.481173  3333 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:35.481707 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 842) took 1257 ms. Trace:
I20260709 12:25:35.482060  3333 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.482113 30990 rpcz_store.cc:276] 0709 12:25:34.224723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.243662 (+ 18939us) service_pool.cc:224] Handling call
0709 12:25:34.244043 (+   381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:34.330901 (+ 86858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.330937 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:34.383022 (+ 52085us) log.cc:844] Serialized 52 byte log entry
0709 12:25:35.036403 (+653381us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:35.036449 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.036530 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.481473 (+444943us) raft_consensus.cc:1685] finished
0709 12:25:35.481495 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.481648 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":226074,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":16260,"thread_start_us":551598,"threads_started":1,"wal-append.queue_time_us":694974}
I20260709 12:25:35.481979  3115 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.485463  3159 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:35.486548 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 230578us
I20260709 12:25:35.488260 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:35.489562 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:35.492674 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 256379us
I20260709 12:25:35.493798 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:35.496491 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:35.419099  3187 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.497678 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2130) took 1264 ms. Trace:
I20260709 12:25:35.470680  3338 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.507881 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:35.501111 31141 rpcz_store.cc:276] 0709 12:25:34.232819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.865135 (+632316us) service_pool.cc:224] Handling call
0709 12:25:34.882999 (+ 17864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.891011 (+  8012us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:34.897016 (+  6005us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.130418 (+233402us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18004dd1c0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:25:35.227336 (+ 96918us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.227599 (+   263us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.241039 (+ 13440us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.281555 (+ 40516us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:35.281829 (+   274us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.313156 (+ 31327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.327225 (+ 14069us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.497492 (+170267us) raft_consensus.cc:1685] finished
0709 12:25:35.497522 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.497627 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31058,"prepare.queue_time_us":39643,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":12185,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":3396,"spinlock_wait_cycles":44822400,"thread_start_us":96454,"threads_started":1,"wal-append.queue_time_us":419456}
I20260709 12:25:35.508514 31060 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:35.507426 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:35.485700  3394 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.510337 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:35.513171 31059 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:35.406783  3413 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.519910  3086 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.520206 31064 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:35.521132 31136 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:35.599561 31060 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:35.510073  3288 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:35.512603  2649 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:35.535656 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 144630us
I20260709 12:25:35.512938  3429 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.214s
I20260709 12:25:35.515998  3296 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.202s
I20260709 12:25:35.546908 30985 raft_consensus.cc:2368] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12.
I20260709 12:25:35.547518 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.714068 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.721078 31138 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:35.719866 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:35.722041 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.528379 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:35.722512 31064 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
W20260709 12:25:35.560544 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.733054 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:35.734452 30991 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:35.535562 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1191) took 1066 ms. Trace:
I20260709 12:25:35.735586 31062 rpcz_store.cc:276] 0709 12:25:34.468863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.469000 (+   137us) service_pool.cc:224] Handling call
0709 12:25:34.469265 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.111442 (+642177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.535315 (+423873us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b54002a3854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:35.535497 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1475229440}
I20260709 12:25:35.738888 30990 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:25:35.739421 30990 raft_consensus.cc:606] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:35.630707  3201 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:35.763312 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:35.765539 31143 raft_consensus.cc:2368] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12.
I20260709 12:25:35.714915  3288 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:35.598704  3187 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:35.630623  3181 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.630714  3249 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.662986  3336 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.758394  2649 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:35.771667 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:35.772279 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:35.773949 31693 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:25:35.784075 30988 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:35.742966  3425 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:35.774971 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:35.776453  3312 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.820s	user 0.005s	sys 0.006s
I20260709 12:25:35.776937  3086 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:35.716686  3413 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.546429  3432 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:35.782385 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.784139 31698 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:35.788084 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:35.789474 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:35.767745  2973 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.788369 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.794257 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:35.798614 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.790580  3312 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.799477 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.800494 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.800809 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:35.801049 30991 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:35.801375 31140 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:35.784112  3181 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.803045  3312 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:35.803952  2649 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:35.804368  3181 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.784439  3187 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.686190  3121 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.804867  2649 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.806713  3187 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:35.807282  3121 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:35.810997  3265 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:35.811587 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:35.812477  3265 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:35.812842  3265 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:35.813190  3265 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:35.814575 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:35.817251 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:35.801941 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:35.778775  3348 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.824087 30988 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:35.818392 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:35.827456  2986 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:35.795926  3432 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.827855  2986 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:35.828260  2986 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.828263 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:35.769562  3407 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:25:35.784566  3249 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.535465  3329 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.826408 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:35.831036 31137 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:25:35.831910 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:35.965613 30988 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:25:36.010994 31136 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
W20260709 12:25:36.010496 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 184178us
I20260709 12:25:35.837802 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:36.011133 30988 raft_consensus.cc:606] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:36.013275 30988 raft_consensus.cc:614] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:36.012012 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:35.831108 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:36.015208 31138 raft_consensus.cc:2368] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
W20260709 12:25:36.017194 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.921439 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:36.014477 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:35.843756 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:36.010486 31137 pending_rounds.cc:85] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 8
I20260709 12:25:36.026448 31137 pending_rounds.cc:107] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:25:36.026758 31137 raft_consensus.cc:2889] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:36.018512 31058 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:36.019466 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:36.041481 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:36.042563 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 67999.5us
I20260709 12:25:35.829360  3432 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.049831 31689 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:35.922489  3358 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.973541  3394 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:35.980931  3442 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:36.062935 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:35.877987  3187 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:35.920743  3438 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:36.037858  3349 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:35.816368  2973 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:35.830832  3429 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:36.129514 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:36.037523  3413 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:35.840837  3332 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:35.790853  3425 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.048763  3437 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:35.789574  3336 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:35.913444  3348 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:36.128592 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 87199.4us
I20260709 12:25:36.141508 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:36.148039 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:36.154644 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:36.156492 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:36.065059  3358 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.167244 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:36.169822 31698 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:36.148186  3332 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:36.108594  3442 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:36.213372  3446 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:25:36.197671  3115 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:36.218523  3115 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:36.218932  3115 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:36.223241  3425 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.196457  3332 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:36.225396  3249 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.226361  3358 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.226706 31696 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:36.209928  3455 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.227602  3455 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.226401 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:36.228861 31058 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:25:36.229704  3387 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.346s	user 0.007s	sys 0.000s
I20260709 12:25:36.230686  3458 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:36.152983  3429 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.232626  3442 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.233501  3373 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:36.233901  3445 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:25:36.236205 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:36.239007 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:36.239909  3323 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:36.240409  3323 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:36.240427 30955 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:36.241329  3323 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.245306  3323 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:36.242640  3181 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:36.246922  3181 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:36.248864  3338 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.249658 31690 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:36.243326 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:36.285816 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:36.240047  3430 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:36.287160 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:36.289147 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:36.302361 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:36.303632 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 809843us
I20260709 12:25:36.135499  3438 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.307240  3281 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
W20260709 12:25:36.316126  3447 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:25:36.317371 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2280) took 1065 ms. Trace:
I20260709 12:25:36.307220  3159 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:36.317736 30983 rpcz_store.cc:276] 0709 12:25:35.252041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.252189 (+   148us) service_pool.cc:224] Handling call
0709 12:25:35.331546 (+ 79357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.414936 (+ 83390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.414967 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.416316 (+  1349us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.718994 (+302678us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b1800330e20. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:36.005260 (+286266us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:36.005302 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.005379 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.317107 (+311728us) raft_consensus.cc:1685] finished
0709 12:25:36.317129 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.317314 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64108,"raft.queue_time_us":104840,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"spinlock_wait_cycles":144280448,"thread_start_us":305007,"threads_started":2,"wal-append.queue_time_us":687901}
I20260709 12:25:36.318981  3159 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:36.319902 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:36.320153  3159 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.296458  3432 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:36.322491  3329 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.322784  3435 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:36.293641  3425 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:36.323377  3435 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:36.323652  3425 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:36.324060  3425 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:36.324290  3435 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:36.327066  2973 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.293459  3430 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.328069  3430 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:36.306964  3249 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:36.331104  3249 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.308121  3134 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:36.331669  3249 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:36.331954 31141 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:36.331869  3134 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.331156 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:36.333328 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:36.297662  3265 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:36.354478  3265 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.354094  3438 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.333993 31065 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:36.321194 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:36.358237 30983 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:36.308326  3320 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.334102  3134 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.334547  3159 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:36.297822  2986 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:36.349963  3135 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:36.335054  3459 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:36.337227  3336 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.386487 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:36.335410  3429 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.375530  3312 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.373636  3291 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:36.390439 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:36.403550 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:36.387382  3414 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:36.392056  3377 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.380949  3320 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.400758  3294 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:36.443200 31062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.000s	sys 0.015s
I20260709 12:25:36.422873  3349 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:36.449190 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:36.450067 31062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.000s	sys 0.015s
I20260709 12:25:36.391150  3135 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:36.459872 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 308759us
I20260709 12:25:36.396816  3459 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:36.480926 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:36.424124  3312 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.501453 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:36.505424 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:36.458853  3134 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:36.459604  3160 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:36.455281  3121 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:36.427310  3291 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.461722  3348 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.514231 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:36.516851 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 113400us
I20260709 12:25:36.514772 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:36.519583 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:36.520109 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:36.455668  3413 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:36.516459 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.443470  3377 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:36.523761  3121 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:36.524513  3312 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.524636  3377 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:36.524915 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:36.525503 31137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:36.526072 31137 raft_consensus.cc:606] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:36.526115  3135 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.442667  3414 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:36.510232  3336 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:36.458000  3294 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.515419  3437 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:36.522723 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.123392 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 603320us
W20260709 12:25:37.123862 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 598975us
I20260709 12:25:37.173390 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:37.175148 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:36.880641 31065 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.002s
I20260709 12:25:37.176244 31065 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:36.517648  3338 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.130303 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:37.213088 31697 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:37.216965 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.217721 31690 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 9: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:25:37.220779 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45746.2us
W20260709 12:25:36.590174  3462 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 12:25:36.643963  3470 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:25:36.680250  3449 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 12:25:36.843631  3425 consensus_meta.cc:287] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.002s
I20260709 12:25:37.223125 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:36.644685  3466 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:25:36.852581  3458 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.004s
W20260709 12:25:36.843286  3430 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.515s	user 0.005s	sys 0.004s
W20260709 12:25:36.972024  3122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.041s	sys 0.037s
W20260709 12:25:36.654176  3288 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.867s	user 0.008s	sys 0.000s
I20260709 12:25:36.527832  3373 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:25:36.850666  3249 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.000s
I20260709 12:25:37.226101 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:36.533476  3134 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:36.680033  3414 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:36.570667  3348 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:37.240391 31696 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:25:36.657346  3291 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.570516  3377 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:36.656876  3332 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:36.517795  3463 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.718s
I20260709 12:25:36.716607  3407 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:37.130832  2986 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.663205  3320 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:36.666538  3336 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:36.843544  3475 raft_consensus.cc:476] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 12:25:36.651019  3459 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:36.597981  3349 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.528810  3115 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:36.843976  3413 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.301374 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2306) took 1972 ms. Trace:
W20260709 12:25:37.320173 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 94154.6us
I20260709 12:25:37.332062 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2304) took 2072 ms. Trace:
I20260709 12:25:37.221419  3464 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:37.282572 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:37.222573  3259 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.005s	sys 0.009s
W20260709 12:25:37.250177  3122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.043s	sys 0.039s
I20260709 12:25:37.278515  3430 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.377466  3480 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:37.278117  3458 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:37.244499  3425 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.274698  3373 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:37.222208  3338 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:37.362694  3323 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.117s	user 0.011s	sys 0.000s
I20260709 12:25:37.284772 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 851) took 1543 ms. Trace:
I20260709 12:25:37.430167 31142 rpcz_store.cc:276] 0709 12:25:35.260133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.325360 (+ 65227us) service_pool.cc:224] Handling call
0709 12:25:35.340964 (+ 15604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.249324 (+908360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.249350 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.292898 (+ 43548us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.640820 (+347922us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:36.640876 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.640982 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.331775 (+690793us) raft_consensus.cc:1685] finished
0709 12:25:37.331802 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.331988 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":908015,"raft.queue_time_us":958388,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":12494,"thread_start_us":280461,"threads_started":1,"wal-append.queue_time_us":960616}
I20260709 12:25:37.320480  3135 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:37.364504  3134 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.427243 31144 rpcz_store.cc:276] 0709 12:25:35.328585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.328738 (+   153us) service_pool.cc:224] Handling call
0709 12:25:35.334278 (+  5540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.244628 (+910350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.244664 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.245124 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.479265 (+234141us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:36.479304 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.479373 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.301133 (+821760us) raft_consensus.cc:1685] finished
0709 12:25:37.301160 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.301303 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":928361,"raft.queue_time_us":960768,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":2276,"thread_start_us":233179,"threads_started":1,"wal-append.queue_time_us":346196}
I20260709 12:25:37.526099 30991 rpcz_store.cc:276] 0709 12:25:35.741401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.979040 (+237639us) service_pool.cc:224] Handling call
0709 12:25:36.038816 (+ 59776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.049297 (+ 10481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.049342 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.064159 (+ 14817us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.428710 (+364551us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:36.428749 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.428819 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.284497 (+855678us) raft_consensus.cc:1685] finished
0709 12:25:37.284525 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.284697 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":7758,"thread_start_us":297850,"threads_started":1,"wal-append.queue_time_us":287039}
I20260709 12:25:37.303041  3294 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.319881  3481 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:37.449055 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:37.546267 30984 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:37.546953 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2529) took 1194 ms. Trace:
I20260709 12:25:37.432763  3320 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:37.445286  3259 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.005s	sys 0.010s
I20260709 12:25:37.423512  3475 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.282021  3249 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.545207 30983 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:37.547309 30984 rpcz_store.cc:276] 0709 12:25:36.354390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.448783 (+1094393us) service_pool.cc:224] Handling call
0709 12:25:37.546885 (+ 98102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.508018  3373 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:37.554687 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:37.419517  3336 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.286690  3288 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.566340 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.567106 31142 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
I20260709 12:25:37.557649 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:37.568681 30984 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:37.559747 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.569265 31141 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 8.
W20260709 12:25:37.551621 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 377269us
I20260709 12:25:37.561628 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2530) took 1207 ms. Trace:
I20260709 12:25:37.426960  3377 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.508221  3414 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:37.418074  3407 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.581101  3187 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:37.564713  3430 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.546144  3134 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:37.574723  3338 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.576742  3249 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.570353 30987 rpcz_store.cc:276] 0709 12:25:36.354714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.554405 (+1199691us) service_pool.cc:224] Handling call
0709 12:25:37.561565 (+  7160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.579952 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:37.597847 30991 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:37.598340 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2534) took 1242 ms. Trace:
I20260709 12:25:37.412567  3437 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.601104 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:37.582124 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 363201us
I20260709 12:25:37.582585 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2535) took 1209 ms. Trace:
I20260709 12:25:37.602691 30984 rpcz_store.cc:276] 0709 12:25:36.372887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.557361 (+1184474us) service_pool.cc:224] Handling call
0709 12:25:37.582528 (+ 25167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.603523 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:37.604023 30984 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:37.554818  3323 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:37.583222 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.011s	sys 0.042s
I20260709 12:25:37.588930 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:37.607288 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.012s	sys 0.042s
I20260709 12:25:37.581542  3187 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:37.594589 31062 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:37.595114 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.598116 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:37.598690 30991 rpcz_store.cc:276] 0709 12:25:36.356901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.557350 (+1200449us) service_pool.cc:224] Handling call
0709 12:25:37.598284 (+ 40934us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.580812  3414 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.600180 31695 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:37.611405 30987 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:37.449431  3480 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.587752  3458 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:37.617136 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.556914  3320 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.608904 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:37.613224 31141 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:37.613747 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2528) took 1273 ms. Trace:
I20260709 12:25:37.618182 30987 rpcz_store.cc:276] 0709 12:25:36.340378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.597826 (+1257448us) service_pool.cc:224] Handling call
0709 12:25:37.613675 (+ 15849us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.615255  3407 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:37.615979 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:37.554577  3475 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:37.590332 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 395418us
I20260709 12:25:37.622494 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:37.615911  3336 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.619588  3414 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.620389 30984 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:37.627066 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2532) took 1271 ms. Trace:
I20260709 12:25:37.627537 30984 rpcz_store.cc:276] 0709 12:25:36.355290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.603319 (+1248029us) service_pool.cc:224] Handling call
0709 12:25:37.626997 (+ 23678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.628881  3283 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.607697 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 358381us
I20260709 12:25:37.629354  3283 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:37.629765 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 499575us
I20260709 12:25:37.631454  3283 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:37.632121  3391 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 2.498s	user 0.020s	sys 0.006s
I20260709 12:25:37.626561  3122 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:37.632767  3122 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.624534 30991 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:37.633172  3377 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.633459 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2536) took 1257 ms. Trace:
I20260709 12:25:37.633814 30991 rpcz_store.cc:276] 0709 12:25:36.376382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.615752 (+1239370us) service_pool.cc:224] Handling call
0709 12:25:37.633405 (+ 17653us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:37.634805  3483 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 12:25:37.635308 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2289) took 1594 ms. Trace:
I20260709 12:25:37.635644 30991 rpcz_store.cc:276] 0709 12:25:36.040947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.634554 (+1593607us) service_pool.cc:224] Handling call
0709 12:25:37.634847 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.635010 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.635034 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.635074 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:37.635110 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.635165 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.635261 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.636538  3187 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.637835  3187 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:37.641100 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 855) took 1593 ms. Trace:
I20260709 12:25:37.626195 30987 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:25:37.634074  3482 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 12:25:37.642835  3425 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.624861 31142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:37.651405 31142 raft_consensus.cc:606] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:37.651752 31142 raft_consensus.cc:614] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:37.624042 31033 leader_election.cc:400] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
I20260709 12:25:37.631588  3320 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:37.653146 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2143) took 1138 ms. Trace:
I20260709 12:25:37.653527 31141 rpcz_store.cc:276] 0709 12:25:36.515004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.625206 (+1110202us) service_pool.cc:224] Handling call
0709 12:25:37.652636 (+ 27430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.652825 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.652850 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.652888 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:37.652926 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.652975 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.653067 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.646449 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:37.654933  3480 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:37.655498 31142 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:37.656049 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2147) took 1134 ms. Trace:
I20260709 12:25:37.656335 31142 rpcz_store.cc:276] 0709 12:25:36.521214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.654936 (+1133722us) service_pool.cc:224] Handling call
0709 12:25:37.655215 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.655822 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.656002 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.657733 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 826) took 1125 ms. Trace:
I20260709 12:25:37.625038  3458 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.644696 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2285) took 1880 ms. Trace:
I20260709 12:25:37.659761 30987 rpcz_store.cc:276] 0709 12:25:35.764357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.625637 (+1861280us) service_pool.cc:224] Handling call
0709 12:25:37.625905 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.644458 (+ 18553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.644640 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.660243  3458 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.661229 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 856) took 1370 ms. Trace:
I20260709 12:25:37.661325  3115 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:37.661618 30987 rpcz_store.cc:276] 0709 12:25:36.290742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.660520 (+1369778us) service_pool.cc:224] Handling call
0709 12:25:37.660767 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.660945 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.660969 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.661007 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:37.661041 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.661088 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.661181 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.663339 30987 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:37.663641  3475 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.663836 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 857) took 1372 ms. Trace:
I20260709 12:25:37.664193  3332 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.647784 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:37.648784 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2142) took 1134 ms. Trace:
I20260709 12:25:37.649008 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.633143  3122 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:37.643833 30991 rpcz_store.cc:276] 0709 12:25:36.047740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.640353 (+1592613us) service_pool.cc:224] Handling call
0709 12:25:37.640640 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.640803 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.640825 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.640860 (+    35us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:37.640895 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.640942 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.641052 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.658028 31142 rpcz_store.cc:276] 0709 12:25:36.532064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.657011 (+1124947us) service_pool.cc:224] Handling call
0709 12:25:37.657278 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.657445 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.657469 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.657507 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:37.657541 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.657594 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.657688 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.654997 31141 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:37.666821 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2145) took 1150 ms. Trace:
I20260709 12:25:37.667141 31141 rpcz_store.cc:276] 0709 12:25:36.516667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.654440 (+1137773us) service_pool.cc:224] Handling call
0709 12:25:37.654715 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.666600 (+ 11885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.666772 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:37.648198 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 43734.1us
I20260709 12:25:37.669641 31139 rpcz_store.cc:276] 0709 12:25:36.514736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.514843 (+   107us) service_pool.cc:224] Handling call
0709 12:25:36.515135 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.515477 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.515504 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.518634 (+  3130us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.021407 (+502773us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:37.604336 (+582929us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.605128 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.605179 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.648588 (+ 43409us) raft_consensus.cc:1685] finished
0709 12:25:37.648607 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.648729 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8815,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":2180,"thread_start_us":754849,"threads_started":2,"wal-append.queue_time_us":809907}
I20260709 12:25:37.652776  3249 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:37.674476  3249 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:37.675385 31694 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 670) took 2338 ms (client timeout 1842 ms). Trace:
W20260709 12:25:37.675782 31694 rpcz_store.cc:269] 0709 12:25:35.337073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.337274 (+   201us) service_pool.cc:224] Handling call
0709 12:25:37.675185 (+2337911us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b540081e7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:37.675286 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201920384}
I20260709 12:25:37.676692  3249 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.626894  3486 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:37.677259  3486 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:37.677642  3486 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:37.664207 30987 rpcz_store.cc:276] 0709 12:25:36.291096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.662940 (+1371844us) service_pool.cc:224] Handling call
0709 12:25:37.663184 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.663629 (+   445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.663791 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.680624 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2524) took 1369 ms. Trace:
I20260709 12:25:37.681002 30987 rpcz_store.cc:276] 0709 12:25:36.311113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.679916 (+1368803us) service_pool.cc:224] Handling call
0709 12:25:37.680187 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.680346 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.680369 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.680408 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:37.680442 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.680490 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.680576 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.682312 30987 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:37.682852 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2525) took 1371 ms. Trace:
I20260709 12:25:37.683305 30987 rpcz_store.cc:276] 0709 12:25:36.311408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.681899 (+1370491us) service_pool.cc:224] Handling call
0709 12:25:37.682145 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.682659 (+   514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.682809 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.685271 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2146) took 1168 ms. Trace:
I20260709 12:25:37.688225 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:37.684517  3401 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:37.690310  3332 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:25:37.695412 31141 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:25:37.695881 31141 pending_rounds.cc:85] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 8
I20260709 12:25:37.696311 31141 pending_rounds.cc:107] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:25:37.696627 31141 raft_consensus.cc:2889] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:37.690086 30991 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:37.685936 31138 rpcz_store.cc:276] 0709 12:25:36.517156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.527112 (+  9956us) service_pool.cc:224] Handling call
0709 12:25:37.647593 (+1120481us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:25:37.685213 (+ 37620us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13686528}
I20260709 12:25:37.691919  3475 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:37.697650 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 859) took 1405 ms. Trace:
I20260709 12:25:37.699572 30991 rpcz_store.cc:276] 0709 12:25:36.291905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.666390 (+1374485us) service_pool.cc:224] Handling call
0709 12:25:37.687649 (+ 21259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.697437 (+  9788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.697600 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.700692 30987 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:37.677912  3134 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.665395  3283 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:37.704519 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2526) took 1392 ms. Trace:
I20260709 12:25:37.705557 30987 rpcz_store.cc:276] 0709 12:25:36.311703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.690643 (+1378940us) service_pool.cc:224] Handling call
0709 12:25:37.690924 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.704305 (+ 13381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.704468 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.708788  3283 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.713474  3283 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.698894  3458 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:37.722201 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 677) took 1949 ms. Trace:
I20260709 12:25:37.722700 31695 rpcz_store.cc:276] 0709 12:25:35.772871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.773011 (+   140us) service_pool.cc:224] Handling call
0709 12:25:35.773283 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.611501 (+1838218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.721908 (+110407us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54006a22d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:37.722129 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4201070464}
I20260709 12:25:37.726506  3288 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:37.728318 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.728969 31694 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:37.730348 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.731873 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.735440  3401 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:37.736488  3401 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.738714 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.739315 31695 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:37.740653 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:37.741191 31062 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:37.685256  3320 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:25:37.743269 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:37.743528 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45512.1us
I20260709 12:25:37.693363  3373 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.750167 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.750955 31060 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:37.751432 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.753527 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:37.756307 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.759138 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:37.765955 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:37.766781 31064 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:37.767910 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.761219 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:37.768131 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.768517 31139 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:37.751526 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:37.749401 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 68671.7us
I20260709 12:25:37.756901 31061 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:37.770113 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:37.771929 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:37.773030 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.774036 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.775436 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.777273 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:37.778724 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.778918 30987 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:37.779306 31138 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:37.779986 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:37.781236 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.781365 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.781586 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.911453 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:37.780007  3430 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.164s
I20260709 12:25:37.782444 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:37.945802 31664 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.655s (SENT) [suppressed 14 similar messages]
I20260709 12:25:37.835656 31058 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:37.947311 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.843056 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 853) took 2031 ms. Trace:
I20260709 12:25:37.831429 31064 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:37.878245  3437 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:37.948402  3288 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:37.815353 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:37.949687  3494 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:37.765465  3458 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.179s
W20260709 12:25:37.814719 30983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2527) took 1474 ms (client timeout 1840 ms). Trace:
I20260709 12:25:37.782171 31061 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:37.750618  3283 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.950789 30983 rpcz_store.cc:269] 0709 12:25:36.340017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.427839 (+ 87822us) service_pool.cc:224] Handling call
0709 12:25:37.814527 (+1386688us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540029aed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:37.814651 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2504863104}
I20260709 12:25:37.781844 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.781443 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.814467 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:37.957918 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.956986 31060 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
W20260709 12:25:37.873276 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 219993us
I20260709 12:25:37.959220 31061 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:37.959718 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.959925  3255 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:37.960112 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.960685 31060 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:37.961067 31061 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:37.961593 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2531) took 1606 ms. Trace:
I20260709 12:25:37.961969 30991 rpcz_store.cc:276] 0709 12:25:36.354971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.700445 (+1345474us) service_pool.cc:224] Handling call
0709 12:25:37.706843 (+  6398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.707201 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:37.707227 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.707630 (+   403us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.950917 (+243287us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:37.950971 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.951031 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.961390 (+ 10359us) raft_consensus.cc:1685] finished
0709 12:25:37.961410 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.961541 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13357,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":206288,"threads_started":1,"wal-append.queue_time_us":248791}
I20260709 12:25:37.964808  3332 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:37.965402  3332 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.965723 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:37.865468 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0622798099998363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 12:25:37.813560  3491 raft_consensus.cc:476] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Not starting pre-election -- already a leader
I20260709 12:25:37.951723 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:37.967272 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 669) took 2635 ms. Trace:
I20260709 12:25:37.971213 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:37.793308 30987 raft_consensus.cc:2368] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
W20260709 12:25:37.779772 31106 connection.cc:256] RPC call timeout handler was delayed by -1.8526204899994809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
I20260709 12:25:37.943862 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:37.975024 31139 raft_consensus.cc:2368] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
I20260709 12:25:37.950090  3494 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:37.958890 31141 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:37.810861  3480 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:37.948854 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:37.778342  3338 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:37.838047  3373 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.850070 31062 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:37.948077 30984 rpcz_store.cc:276] 0709 12:25:35.811751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.628323 (+1816572us) service_pool.cc:224] Handling call
0709 12:25:37.628627 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.628839 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.628863 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.629332 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.740991 (+111659us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:37.741023 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.741157 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.842854 (+101697us) raft_consensus.cc:1685] finished
0709 12:25:37.842875 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.843002 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":111285,"threads_started":1,"wal-append.queue_time_us":209647}
I20260709 12:25:37.775275  3292 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:37.835911  3490 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.971412 31692 rpcz_store.cc:276] 0709 12:25:35.331676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.331907 (+   231us) service_pool.cc:224] Handling call
0709 12:25:35.332244 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.416530 (+ 84286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.967000 (+2550470us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b54006ee6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:37.967182 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113437568}
I20260709 12:25:37.949169  3437 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.778110  3475 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:37.983804  3292 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.986897  3475 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:37.987007  3373 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:37.987282  3292 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:37.987458  3475 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:37.987629  3373 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.989701  3373 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:37.991819  3437 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.822268  3413 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:37.996726  3413 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:37.976173 31106 connection.cc:256] RPC call timeout handler was delayed by -1.0033669419995022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
I20260709 12:25:37.998749  3332 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.821889 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:37.972932  3456 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:25:37.999291 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:38.001259  3494 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.990149  3373 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.002349  3494 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.002871 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:38.003757 31106 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:37.994633  3134 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.973973  3432 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:37.995427  3292 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:37.999532 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 177753us
I20260709 12:25:38.006378  3432 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.006791  3432 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.008044  3292 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.910976  3349 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:38.008687 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:38.008984  3349 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.980616 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:37.998718  3413 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:37.979462 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:37.973245 31061 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.002296  3414 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:38.001485  3475 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.005040 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:38.000598  3371 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:37.977371  3320 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.011817 31058 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:37.983399 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:38.044140 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.087170 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.122750 31062 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:37.781137  3122 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:37.978246  3480 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:38.124743 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:38.125469 31695 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:38.126906 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 115423us
I20260709 12:25:37.967613  3283 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.129259  3486 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:37.981994  3491 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.973330  3255 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 12:25:37.981305  3336 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:38.134951 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 123114us
I20260709 12:25:38.132841 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:38.024789  3377 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:38.023612  3135 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:38.013442  3459 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.174173 31064 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:38.174659 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:38.160306 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.176111 31139 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:38.178023  3486 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.179522 31136 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:38.120350  3371 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:38.192909 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:37.980734  3338 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.199137 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.024448  3294 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.156s
I20260709 12:25:38.033603  3134 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.151s
I20260709 12:25:38.131444  3480 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:38.207026 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.207643 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.208032 31695 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:38.210988 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:38.179677  3135 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.010735  3490 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.201749  3371 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:38.236123 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 830) took 1382 ms. Trace:
I20260709 12:25:38.250931 31142 rpcz_store.cc:276] 0709 12:25:36.853567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.682316 (+828749us) service_pool.cc:224] Handling call
0709 12:25:37.682574 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.682755 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:37.911290 (+228535us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.235739 (+324449us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.235796 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:38.235848 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.235918 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.236055 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6943,"prepare.queue_time_us":25175,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":120223,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":46219,"thread_start_us":538048,"threads_started":3,"wal-append.queue_time_us":217036}
I20260709 12:25:38.023417  3373 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.133802  3491 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.266901 31065 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:38.197328  3459 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.270506 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.274683 30986 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:38.126976  3320 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.235881  3135 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.218571  3377 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:38.275700 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:38.291219 31060 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:25:38.270393  3509 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:38.309921  3514 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:38.310470  3514 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:38.311472  3514 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.313669  3475 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.324415 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1206) took 2543 ms. Trace:
I20260709 12:25:38.331362 31067 rpcz_store.cc:276] 0709 12:25:35.781350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.781546 (+   196us) service_pool.cc:224] Handling call
0709 12:25:35.781831 (+   285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:35.795160 (+ 13329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.795206 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:35.795900 (+   694us) log.cc:844] Serialized 52 byte log entry
0709 12:25:37.845124 (+2049224us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b1800413b60. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:38.013295 (+168171us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:38.013333 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.013551 (+   218us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.323980 (+310429us) raft_consensus.cc:1685] finished
0709 12:25:38.323999 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.324298 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23951,"raft.queue_time_us":13952,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":3621632,"thread_start_us":167889,"threads_started":1,"wal-append.queue_time_us":2487049}
I20260709 12:25:38.353381 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:38.354035 31135 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.299988  3513 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.370318  3513 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.370715  3513 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.381407 31135 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:38.293502  3377 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.192984  3336 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:38.393690 31135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 824) took 1860 ms (client timeout 1610 ms). Trace:
I20260709 12:25:38.432957  3336 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:38.433321  3467 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.921s	user 0.010s	sys 0.006s
W20260709 12:25:38.431088 31135 rpcz_store.cc:269] 0709 12:25:36.527326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.527404 (+    78us) service_pool.cc:224] Handling call
0709 12:25:38.353233 (+1825829us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:25:38.382059 (+ 28826us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20835712}
I20260709 12:25:38.457183 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:38.458242 31137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 825) took 1930 ms (client timeout 1761 ms). Trace:
W20260709 12:25:38.458698 31137 rpcz_store.cc:269] 0709 12:25:36.527628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.716203 (+188575us) service_pool.cc:224] Handling call
0709 12:25:38.457002 (+1740799us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:25:38.458143 (+  1141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13707008}
I20260709 12:25:38.460485 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2144) took 1945 ms. Trace:
I20260709 12:25:38.460980 31136 rpcz_store.cc:276] 0709 12:25:36.515233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.527546 (+ 12313us) service_pool.cc:224] Handling call
0709 12:25:37.648830 (+1121284us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:25:38.460247 (+811417us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b540006efd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:38.460417 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190203648}
W20260709 12:25:38.484290 31143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2307) took 3151 ms (client timeout 1722 ms). Trace:
W20260709 12:25:38.484787 31143 rpcz_store.cc:269] 0709 12:25:35.332708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.452124 (+119416us) service_pool.cc:224] Handling call
0709 12:25:38.484053 (+3031929us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b540006fed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:38.484191 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718420352}
W20260709 12:25:38.405437  3504 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 12:25:38.323062  3320 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.511163 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2303) took 2996 ms. Trace:
I20260709 12:25:38.511602 31059 rpcz_store.cc:276] 0709 12:25:35.514709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.514912 (+   203us) service_pool.cc:224] Handling call
0709 12:25:35.515943 (+  1031us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.717152 (+201209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.717182 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.717616 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.770592 (+ 52976us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:35.770627 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.510775 (+2740148us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54003962d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:38.510827 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.510875 (+    48us) raft_consensus.cc:1685] finished
0709 12:25:38.510888 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.511105 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":461856128,"thread_start_us":52577,"threads_started":1,"wal-append.queue_time_us":150249,"wal-append.run_cpu_time_us":2640,"wal-append.run_wall_time_us":1319312}
W20260709 12:25:38.516731 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 388539us
I20260709 12:25:38.517443 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:38.519109 31141 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:25:38.520447 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.521704 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.523922 31109 leader_election.cc:400] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:38.355912  3432 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.524221 31109 leader_election.cc:403] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:38.526458 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:38.526916 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.525118 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:38.532820 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.533207 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2617) took 3641 ms. Trace:
I20260709 12:25:38.493211  3455 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:38.518652  3514 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:38.565557 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.566359 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.566757 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.567411 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:38.567916 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.569181 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:38.558583 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 361535us
I20260709 12:25:38.570528 31693 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:38.523746  3122 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:38.524058  3494 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
W20260709 12:25:38.559828 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 343769us
I20260709 12:25:38.551234  3320 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:38.552192 31063 rpcz_store.cc:276] 0709 12:25:34.892301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.892459 (+   158us) service_pool.cc:224] Handling call
0709 12:25:34.897212 (+  4753us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.905949 (+  8737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.905978 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.906467 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.202424 (+295957us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:35.202476 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.304628 (+3102152us) spinlock_profiling.cc:243] Waited 8.41 ms on lock 0x7b5400385254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:38.304697 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.304762 (+    65us) raft_consensus.cc:1685] finished
0709 12:25:38.304779 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.533151 (+228372us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22707,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":19350400,"thread_start_us":264577,"threads_started":1,"wal-append.queue_time_us":524194,"wal-append.run_cpu_time_us":1473,"wal-append.run_wall_time_us":1002119}
I20260709 12:25:38.572629 30983 raft_consensus.cc:2368] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:38.572139 30986 raft_consensus.cc:2368] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
I20260709 12:25:38.570763 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.574469 31698 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:38.526584  3283 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:38.516532  3349 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.561048  3490 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.576181 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:38.550181  3348 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:38.578945 31691 raft_consensus.cc:2410] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:25:38.562459  3487 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:25:38.588896 31061 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.565s	user 0.005s	sys 0.005s
I20260709 12:25:38.589275 31061 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:38.579038 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.579383 31694 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:38.562990  2973 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.590997 30986 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:38.592445 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.569799  3338 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:38.593636 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.562232  3515 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.583163  3283 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.576170  3320 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:38.600449 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:38.600559 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:38.600704 31697 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:38.605746 30986 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:38.601363 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.593884 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:38.608456 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:38.609484 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.609652 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:38.610762 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:38.611776 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.614473 31690 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 12:25:38.614583 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.599774  3128 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:38.593012  3348 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.627154 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.633206 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:38.596797  3459 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.636834 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:38.616484 31697 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:38.640792 30989 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:38.642495  3516 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.157s	user 0.000s	sys 0.005s
I20260709 12:25:38.642439  3128 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:38.643254 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.643623 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.653904 31694 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:38.654359 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.644234 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.645395  3513 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.617862  3520 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:38.647966  3348 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:38.595034  3517 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:25:38.615829  3134 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:38.616784  3475 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:38.617871  3518 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:25:38.641459  3401 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:38.651234  3432 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:38.652276 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:38.655272 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:38.691828 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.691962 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:38.692413 30984 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:38.693264 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.669025 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.607402  3320 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:38.616376  3122 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:38.595320  3487 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.688107 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.595206  2973 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.653074  3402 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.653239  3338 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:25:38.694955 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.694608 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:38.694767 30989 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:38.715854 31694 raft_consensus.cc:2376] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14. Re-sending same reply.
I20260709 12:25:38.694159 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.716188 31033 leader_election.cc:400] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
I20260709 12:25:38.733505 30992 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:38.734043 31033 leader_election.cc:403] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:38.602962  3283 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.686136  3475 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:38.737079 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 119882us
I20260709 12:25:38.740172 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:38.742640 31140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2305) took 3426 ms (client timeout 1626 ms). Trace:
W20260709 12:25:38.743163 31140 rpcz_store.cc:269] 0709 12:25:35.316142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.316804 (+   662us) service_pool.cc:224] Handling call
0709 12:25:38.742402 (+3425598us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54000be5d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:38.742553 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1111761280}
I20260709 12:25:38.740968 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:38.746411 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:38.735445  3459 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
I20260709 12:25:38.747227 31106 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:38.692895  3373 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.684233  3349 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:25:38.693426  3473 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:38.703282  3348 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:38.758883 30987 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:38.703850  3513 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.753579 30986 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:38.694559  3401 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:25:38.733385  3490 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:38.756067 31106 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:38.767119 30984 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:38.752768  3518 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:38.698622  3480 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.738014  3338 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.707876  3135 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.753189  3086 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.747880  3475 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:38.772676 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:38.749704  3336 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:25:38.780573 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 674) took 3049 ms. Trace:
I20260709 12:25:38.754959  3134 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:38.754784  3511 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.772257  3526 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.763393  3473 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:38.781013 31689 rpcz_store.cc:276] 0709 12:25:35.731414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.731598 (+   184us) service_pool.cc:224] Handling call
0709 12:25:35.731902 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.060240 (+328338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.780293 (+2720053us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b540055c654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:38.780515 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730665728}
I20260709 12:25:38.771677  3518 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.758291  3413 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:25:38.799410  3473 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.787571 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:38.778571  3371 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 12:25:38.764537  3459 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:25:38.803632 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:38.805182 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 119730us
I20260709 12:25:38.808642 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2283) took 3038 ms. Trace:
I20260709 12:25:38.757634  3512 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:25:38.815860  2973 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.821411  3515 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:38.778894  3338 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.782343  3086 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.778043  3487 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.783623  3134 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.789716  3513 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:38.817929  3402 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:38.900136 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:38.899628 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 145687us
I20260709 12:25:38.900786 31063 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.805131  3518 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.825229 30990 rpcz_store.cc:276] 0709 12:25:35.763460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.057900 (+294440us) service_pool.cc:224] Handling call
0709 12:25:36.058283 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.156998 (+ 98715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.157041 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.157581 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.388167 (+230586us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:36.388204 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.432770 (+2044566us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:38.801436 (+368666us) spinlock_profiling.cc:243] Waited 98.2 ms on lock 0x7b5400226854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:38.801510 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.801570 (+    60us) raft_consensus.cc:1685] finished
0709 12:25:38.801589 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.801767 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8449,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":1103,"spinlock_wait_cycles":228582528,"thread_start_us":191225,"threads_started":1,"wal-append.queue_time_us":269724}
I20260709 12:25:38.832500  3526 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:38.834419  3473 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:38.906440 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.810326  3371 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.913542 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:38.830348  3401 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:25:38.835767  3160 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.914165 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:38.897719  3086 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.895645  3377 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:38.900004  3134 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.927637 31063 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:38.900338  3513 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:38.934592 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 867) took 1113 ms. Trace:
I20260709 12:25:38.935127 30986 rpcz_store.cc:276] 0709 12:25:37.821353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.767601 (+946248us) service_pool.cc:224] Handling call
0709 12:25:38.767870 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.768126 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:38.839680 (+ 71554us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.934148 (+ 94468us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.934212 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:38.934293 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.934379 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.934517 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7780,"prepare.queue_time_us":156785,"thread_start_us":152952,"threads_started":2}
I20260709 12:25:38.926082  3160 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:38.947307 31696 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:38.954803 31697 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:25:38.955724  3413 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:38.956674  3413 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:38.957015  3413 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:38.919201  3526 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.959157  3533 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:38.917841  3349 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:25:38.939389  3513 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.980602 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 852) took 3238 ms. Trace:
I20260709 12:25:38.981011 30988 rpcz_store.cc:276] 0709 12:25:35.741706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.038287 (+296581us) service_pool.cc:224] Handling call
0709 12:25:36.038596 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.170637 (+132041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.170669 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:36.170711 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:36.170750 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.980381 (+2809631us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400224a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:38.980426 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.980546 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303208704}
I20260709 12:25:38.974714  3486 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:38.994506 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.999362 31063 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:38.983104  3283 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:38.938566  3377 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:38.957897  3413 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.982892  3135 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:39.005028  3529 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 12:25:39.005409  3528 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:25:39.006805  3532 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:38.993854  3480 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:39.005033  3536 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:39.019454  3526 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:39.022708  3475 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:38.994170  3486 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:39.101516 30989 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:39.101568 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.105080 31144 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:39.105556 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.051525 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 862) took 1231 ms. Trace:
I20260709 12:25:39.049266 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.026176 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:39.106360 30992 rpcz_store.cc:276] 0709 12:25:37.819891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.757141 (+937250us) service_pool.cc:224] Handling call
0709 12:25:38.767119 (+  9978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.911031 (+143912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.911058 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.911095 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:38.911128 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.051306 (+140178us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54001f7054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:39.051347 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.051473 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330492160}
I20260709 12:25:39.107457 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:39.107707 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58531.8us
I20260709 12:25:39.043702 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:39.108940 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.109833 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.110193 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:25:39.108116 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44225.7us
I20260709 12:25:39.110975 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:39.110951 31142 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:39.112388 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.113870 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.114089 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.094543 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.117785 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.118702 31139 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:39.114609 31142 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:39.120532 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.121234 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.121753 31142 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:39.092146 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:39.123889 30986 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:39.079986 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.076956 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.087002 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.127588 31143 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:39.128659 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.130043 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.036140  3464 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:39.136284 31033 leader_election.cc:400] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13.
I20260709 12:25:39.136284 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.170914  3464 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:39.171375 31136 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:39.171756  3464 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.172940  3464 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.172991  3348 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:39.174400 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:39.175000 31063 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.175031  3086 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.175536 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:39.134940  2989 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:39.046350  3413 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.177201  3086 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:39.144016  3533 raft_consensus.cc:2764] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:39.128777  3349 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:39.142733  3490 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:39.173017 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:39.178411  3490 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.178809  3490 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.179224 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.179631  3533 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:39.179946 31062 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:39.180096  3533 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.180517  3533 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.119851  3525 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:39.182678  3525 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.184717  3525 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.037943  3333 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:39.188058  3474 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:39.188898  3474 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:39.189229  3474 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:39.058818  2970 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.190361  3323 raft_consensus.cc:993] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:39.190874  3323 raft_consensus.cc:1081] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:39.179267 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.192492 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.192481 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.105540  3160 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.195304 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:39.146770 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.195600  3160 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.196348 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:39.137262  3291 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:39.197016  3291 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.197441 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.197887  3520 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:39.199522  3128 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:39.199898  3128 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:39.199054  3291 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.137513  3292 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.200771  3128 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.201530  3292 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.189992  3474 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.203559  3292 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.035720  3134 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.204169  3474 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.204316 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.204954 31060 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:39.148468 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:39.205644  3134 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.147717  3250 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:39.149881 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:39.207139 31692 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:39.207367 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.207592 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.144539  2915 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:39.196368 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:39.209640 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.209976  3381 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:39.150560 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.210563 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:39.210626  3381 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:39.211783 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.212059 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:39.212543 31698 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:39.212282  3381 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.158494  3402 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:39.044513  3377 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:39.036480  3338 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.043041  3371 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:39.211380 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.161513 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2332) took 1390 ms. Trace:
I20260709 12:25:39.149654 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.215236 31141 rpcz_store.cc:276] 0709 12:25:37.770856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.004271 (+233415us) service_pool.cc:224] Handling call
0709 12:25:38.518882 (+514611us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.519520 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.161320 (+641800us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b540006efd0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:39.161455 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055855360}
I20260709 12:25:39.215956 31139 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:39.216508 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.217209 31141 raft_consensus.cc:2368] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12.
I20260709 12:25:39.166821 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.105468  3517 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.219156 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.219759 31137 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:39.219611  3517 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.220610 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.221880  3517 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.025094  3475 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.061527  3372 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:39.222602  3372 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:39.215247 31693 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:39.221732 31692 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:39.223541  3372 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.224042  3475 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:39.224563  3372 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:39.225153 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:39.053148  3320 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.226531 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.225324 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:39.228785  3348 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:39.229696  3320 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.101471  3515 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.172150 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.230563  3250 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:39.175622 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.047500  3486 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.230580  3515 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.231415  3250 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:39.231572 31059 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:39.190485  2970 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.119477  3518 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.210625 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.232510 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.232883 31694 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:39.233371 31059 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:39.234045  3518 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.234347 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:39.225983  3201 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:39.101585  3480 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:39.232375  3250 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.235317  3201 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:39.235901  3480 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.237322  3486 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:39.235953  3201 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:39.217036 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.242074 31139 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:39.243010 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.243615 31138 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:39.244282 31139 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:39.245888  3201 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.218096 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.254252  3515 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.254599 31141 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:39.246228  2970 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:39.246618  3128 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.169234  3513 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.250556 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:39.256124 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.257010 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:39.138375 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.279034 31144 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:39.256366 31141 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:39.280820 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.281883 31141 raft_consensus.cc:2410] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:25:39.126094 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.269304 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:39.165208 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.394598 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.394841 31695 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:25:39.395170 31143 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:39.271967 31137 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
W20260709 12:25:39.396026 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 139976us
I20260709 12:25:39.265862 31063 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:39.275626 31136 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:39.276715 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.396021 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.398936 31143 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:39.323482  3348 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:39.403748 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.404467 31143 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:39.404752 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 188514us
W20260709 12:25:39.405380 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 138275us
I20260709 12:25:39.238055  3480 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:39.303957 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.386411 31138 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:39.286369  3201 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:39.390658 31142 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:39.285944  3524 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:39.415998 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:39.274834  3474 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.399781 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.400344 31140 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:25:39.381073  3435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.016s	sys 0.000s
I20260709 12:25:39.401906 31144 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:25:39.397071 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:39.422809 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.427039 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.402627 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.415731 30989 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:39.400521 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 150028us
I20260709 12:25:39.418673 30985 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:39.255458  3533 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.442849  3533 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.417402  3290 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:39.404467  3402 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:39.447914 31068 raft_consensus.cc:2410] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:39.447808  3294 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:39.448477  3294 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:39.449086 31068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 1199) took 3961 ms (client timeout 1761 ms). Trace:
W20260709 12:25:39.449470 31068 rpcz_store.cc:269] 0709 12:25:35.487895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.488032 (+   137us) service_pool.cc:224] Handling call
0709 12:25:39.448873 (+3960841us) spinlock_profiling.cc:243] Waited 3.96 s on lock 0x7b54003976d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:39.448987 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9104653056}
I20260709 12:25:39.422809 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:39.458310 31063 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.458155 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.423094 31135 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:39.459036 31144 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:39.425233 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:39.430671 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:39.428467 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:39.440538 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.450294  3411 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.451452 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.452638 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.455693 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 863) took 1635 ms. Trace:
I20260709 12:25:39.457057 31067 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:39.403870  2989 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:39.454123 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:39.466852 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.421790 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:39.430047 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:39.419334 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.463554 30983 rpcz_store.cc:276] 0709 12:25:37.820198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.765269 (+945071us) service_pool.cc:224] Handling call
0709 12:25:39.055051 (+289782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.055246 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:39.055533 (+   287us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.248690 (+193157us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.248858 (+   168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.248902 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:39.248960 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.249042 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.455537 (+206495us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54002c1d50. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:39.455638 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3579,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":56,"raft.queue_time_us":767,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":168,"spinlock_wait_cycles":658400896,"thread_start_us":192652,"threads_started":1,"wal-append.queue_time_us":192980}
I20260709 12:25:39.467837 31144 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:39.408341  3439 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:39.470046 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 50800.1us
W20260709 12:25:39.430017  3435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.017s	sys 0.000s
I20260709 12:25:39.469933 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.483690 31137 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:25:39.461053 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 64067.6us
I20260709 12:25:39.416642  3486 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:39.480149 31661 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:39.481484 30985 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:39.485101 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:39.487498 31067 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:39.487767 31139 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:39.488238 31065 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:39.489519 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:39.491979 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.421478  3524 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.497656 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:39.467056  3201 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:39.501928 31140 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:39.502095 30985 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:39.473412  3250 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:39.503108 31144 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:39.419234  3348 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.505180 31691 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:39.505158 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.506031 31063 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:39.506606 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:39.429656  3474 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.508284 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.483045  3283 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:39.539505 31140 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.510527 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:39.449499  3294 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.525126 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:39.517001 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.490607  3288 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:39.484894  3490 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.470868  3411 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:39.486384  3439 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.516366 30991 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:39.593992 31141 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:39.513618 31137 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
W20260709 12:25:39.733497 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 208453us
I20260709 12:25:39.496116  3354 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:39.603037 31140 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:39.493925  2915 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:39.543355 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:39.562045 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:39.577447 31063 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:39.740830 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 230405us
W20260709 12:25:39.656522 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.780169 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.787423 31138 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:39.508281  3250 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.538976  3290 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:39.813657 31063 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:39.503909  3533 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:39.514274  3160 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:39.815452 31062 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:39.504338  3201 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:39.743197  3485 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:39.736910  3288 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:39.740446  3484 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.834110 31698 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:39.514608  3511 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:39.775602  3413 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:39.814965  3401 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.826958 31138 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:39.838522 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:39.829505 31063 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:39.779270  3354 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:39.583792  3524 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.563932  3202 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.276s
I20260709 12:25:39.830571  3201 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:39.823905  3160 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:39.866400  3160 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:39.866653  3546 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
W20260709 12:25:39.867543  3543 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
I20260709 12:25:39.859130  3354 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:39.869992  3547 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
I20260709 12:25:39.823212  3294 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:39.771854  3283 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.840118  3484 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.836889  3288 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:39.863678  3202 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:39.876986 31142 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:39.818704  3373 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:25:39.893206 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2539) took 2147 ms. Trace:
W20260709 12:25:39.893376 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 156738us
I20260709 12:25:39.895464 31033 leader_election.cc:400] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12.
I20260709 12:25:39.895833 31033 leader_election.cc:403] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:39.835336  2970 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.818902  3250 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:39.835613  3533 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:39.889918 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.893713 30984 rpcz_store.cc:276] 0709 12:25:37.745873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.736371 (+990498us) service_pool.cc:224] Handling call
0709 12:25:38.766958 (+ 30587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.770353 (+  3395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.892973 (+1122620us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b54000762d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:39.893135 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69588736}
I20260709 12:25:39.910270 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.860780  3513 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:39.910452 31142 raft_consensus.cc:2368] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14.
I20260709 12:25:39.851087 31063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
W20260709 12:25:39.941287 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45991us
I20260709 12:25:39.942229 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:39.943558 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:39.944141 31140 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:39.948536 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:39.966550 31068 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:39.968909 31142 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:39.855020  3511 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.971339 31136 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:39.888154  3202 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:39.992934 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:39.970482  3545 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:25:39.909205  3201 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.872526  3401 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:39.943499  3290 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:40.009321 31063 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:40.009974 31063 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:39.887850  3524 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 12:25:39.970947  3474 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.048322 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.049340 31142 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:40.049356 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.056679 31063 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.083740 31065 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
W20260709 12:25:40.147038 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.001226  3348 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:39.941355  2970 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:40.027137  3349 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.849s	user 0.010s	sys 0.000s
I20260709 12:25:40.165129 31140 raft_consensus.cc:2410] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:25:40.172199 31664 leader_election.cc:336] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.603s (SENT)
I20260709 12:25:40.182472 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:40.183348 31664 leader_election.cc:336] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.610s (SENT)
W20260709 12:25:40.172574 31695 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.024s	sys 0.103s
W20260709 12:25:40.184295 31695 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.024s	sys 0.103s
I20260709 12:25:40.010391  3373 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12.
I20260709 12:25:40.188860  3349 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:40.192173 31140 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:40.193287 31135 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 12:25:40.199889 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.239200  3379 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:40.240099  3379 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:40.240324  3511 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.240717  3379 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.243695  3379 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.297377 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 868) took 1201 ms. Trace:
I20260709 12:25:40.297945 30986 rpcz_store.cc:276] 0709 12:25:39.095914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.163141 (+ 67227us) service_pool.cc:224] Handling call
0709 12:25:39.163408 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.163698 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.163724 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.164206 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.492645 (+328439us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:39.492679 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.492766 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.297116 (+804350us) raft_consensus.cc:1685] finished
0709 12:25:40.297145 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.297302 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1205,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":328122,"threads_started":1,"wal-append.queue_time_us":432417}
I20260709 12:25:40.184266  3348 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:40.182605  3513 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:40.190465  3490 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:40.335664 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.254802  3439 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.050295  3533 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.370100 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.355191  2915 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.373556  3379 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:40.190284  3202 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:40.370090  3160 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.001s
W20260709 12:25:40.397962 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 455815us
W20260709 12:25:40.393236  3464 consensus_meta.cc:287] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.220s	user 0.012s	sys 0.000s
I20260709 12:25:40.401748 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:40.398479 31140 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:25:40.382565  3201 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.355538  3513 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:40.393749  3354 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
I20260709 12:25:40.425743 31059 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:25:40.426487 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.080s	sys 0.089s
I20260709 12:25:40.420039 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:40.435549 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.081s	sys 0.090s
W20260709 12:25:40.436043 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 615580us
I20260709 12:25:40.348517  3290 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:40.401156  3555 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
W20260709 12:25:40.401120  3558 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 12:25:40.444090 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.442729 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:40.401262  3554 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 12:25:40.398499  3459 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.398833  3202 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:40.401458  3561 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.332s	user 0.000s	sys 0.000s
I20260709 12:25:40.401333  3557 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.450363 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:40.451023 31058 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:40.452163 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2655) took 1256 ms. Trace:
I20260709 12:25:40.452600 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.395139  2915 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.481616 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:40.483282 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.484687 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.486920 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:40.453338 31060 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:40.454072 31067 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:40.454453  3562 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:25:40.455387  3565 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:25:40.438308  2970 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.460403  3358 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:40.471967 31106 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:40.401063  3484 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.489192 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:40.476529  3201 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:40.437934  3283 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.401417  3559 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 12:25:40.479902 31064 rpcz_store.cc:276] 0709 12:25:39.195792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.448942 (+253150us) service_pool.cc:224] Handling call
0709 12:25:39.449363 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.458852 (+  9489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.459029 (+   177us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.485668 (+ 26639us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.789118 (+303450us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:39.789154 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.789226 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.451903 (+662677us) raft_consensus.cc:1685] finished
0709 12:25:40.451930 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.452090 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8482,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":8022,"thread_start_us":287290,"threads_started":1,"wal-append.queue_time_us":392555}
I20260709 12:25:40.479071  3557 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.507341 31064 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:40.492614 31058 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:40.457218 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2327) took 1271 ms. Trace:
I20260709 12:25:40.494834 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:40.396484  3160 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.440743  3379 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:25:40.439157  3564 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:40.480964  3372 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.256s	user 0.010s	sys 0.001s
I20260709 12:25:40.449270  3459 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:40.508445 31062 rpcz_store.cc:276] 0709 12:25:39.185520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.194535 (+  9015us) service_pool.cc:224] Handling call
0709 12:25:39.194811 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.823970 (+629159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.456909 (+632939us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54000c7454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:40.457151 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426726784}
I20260709 12:25:40.500845 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:40.437480  3566 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 12:25:40.442700  3354 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.527159 31061 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:40.499372  3475 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:40.534729 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:40.496332  3485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.049s	sys 0.080s
I20260709 12:25:40.535872 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:40.503300  3086 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:40.502447  3358 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:40.538499  3511 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:40.539059  2915 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.539989  3181 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:40.542608  3459 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:40.543393 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.543823  3338 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:40.535157  3401 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.545686  3557 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.546406  3373 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:40.548836 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.506760  3349 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:40.526719  3564 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:40.502256  3381 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 12:25:40.567495 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 208997us
W20260709 12:25:40.568620 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 390887us
I20260709 12:25:40.571257 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:40.534255  3201 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:40.577222 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:40.579093 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:40.580683 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:40.600409 30986 raft_consensus.cc:2410] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 12:25:40.600528 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 65380.6us
W20260709 12:25:40.582074 31664 leader_election.cc:336] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.655s (SENT)
I20260709 12:25:40.602082 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:40.583837 31060 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:40.509968  3464 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 12:25:40.551542  3485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.050s	sys 0.080s
I20260709 12:25:40.555559  3086 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:40.604388 30984 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:40.599646 30991 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:40.607336 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.608179 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.629238 31142 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:25:40.584142  3475 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:40.585561  3250 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.001s
W20260709 12:25:40.620154 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 62781us
I20260709 12:25:40.618561  3485 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:40.618490  3557 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:25:40.598621  3281 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:40.624197 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.646126 31143 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:40.625214  3381 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:40.648756 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:40.566852  3511 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:40.625947 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:40.651757 31144 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:40.652364 31136 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 13: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:40.632548 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:40.633339 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:40.652951 30985 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:40.653013 30990 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:40.633714 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:40.632753 30986 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:40.633108 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:40.654290 31137 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:25:40.654692 31137 pending_rounds.cc:85] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 10
I20260709 12:25:40.654908 31137 pending_rounds.cc:107] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 13.11
W20260709 12:25:40.638439 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 130545us
W20260709 12:25:40.655959 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.642078 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:40.587764  3564 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.572322  3571 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:40.599088  3373 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:40.642989 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.605286  3349 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:40.647095 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.663628 31141 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:40.663954 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:40.664397 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.649281  3281 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.629750 31058 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:40.633935  3086 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.655143 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.655184 31137 raft_consensus.cc:2889] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:40.651553 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.635517  3569 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.636855  3568 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:40.670670  3569 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.670853  3568 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.673753  3569 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.674978  3568 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.644682  3566 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:25:40.659689  3585 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.642616  3576 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:40.678637  3585 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.627053  3570 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:25:40.640255  3572 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:25:40.679703  3570 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.679975  3572 consensus_queue.cc:1243] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:40.643045  3475 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.640283  3181 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:40.681936 31143 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:40.683548 31137 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:40.684151 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.651165  3381 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:40.630091  3358 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.681278  3160 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.704700  3358 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:40.707036  3349 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.686146 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.650646  3557 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:40.701383 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:40.644709  3250 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.711015  3287 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:40.715072  3588 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:40.715276  3287 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.716226  3250 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.717334  3287 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.702628 31144 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:40.710286 31141 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 9.
I20260709 12:25:40.711596  3557 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:40.718173  3475 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.718536  3511 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.719014  3557 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.720168  3557 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:40.705969  3566 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:40.723158  3381 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:40.725262  3294 consensus_meta.cc:287] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.840s	user 0.009s	sys 0.000s
I20260709 12:25:40.724961  3086 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.711710  3578 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.736753  3578 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.739257  3578 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:40.728699  3288 consensus_meta.cc:287] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.790s	user 0.011s	sys 0.000s
W20260709 12:25:40.701710 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 96039.6us
W20260709 12:25:40.749130 31664 leader_election.cc:336] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.761s (SENT)
I20260709 12:25:40.748960  3288 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.749557 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:40.750360  3557 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:40.750560 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.734828  3485 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:40.751518 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.726091  3584 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.752068  3584 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.752924  3379 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:40.754136  3287 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:40.732189  3373 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.755096  3379 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:40.703253  3513 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.680121  3570 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:40.710306  3181 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.687098  3354 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.711740  3160 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:40.710481  3201 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.758641  3570 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.759685  3181 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.760241  3570 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.731628  3575 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:40.740887  3281 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.756108  3379 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:40.761638  3575 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:40.762871  3381 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:40.763087  3379 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.763285  3575 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.764297  3379 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:40.770104  3584 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.781860  3571 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:40.633309 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:40.815062 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 181617us
I20260709 12:25:40.821369 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:40.824108 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:40.826444 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:40.828358 30990 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:40.829810 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:40.831064 30992 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:40.846984 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 190137us
W20260709 12:25:40.845764 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 181741us
I20260709 12:25:40.847592 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.887264 31064 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:40.889251 31064 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:40.848788 31137 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:40.859175 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:40.754431 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:40.892112 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:40.892756 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.893179 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:40.861673  3181 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:40.761492  3288 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.869663  3086 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.895824  3358 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:40.898818  3348 consensus_meta.cc:287] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.006s
I20260709 12:25:40.785756  3564 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.899360  3250 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:40.899478  3348 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:40.900050  3250 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:40.900547  3250 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:40.901806  3564 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.906317  3570 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:40.906486  3571 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:40.893926 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 139445us
I20260709 12:25:40.909665  3591 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:40.909359  3571 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.910267  3582 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:25:40.907063  3485 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:40.912501  3571 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.913818 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.914748 31058 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:40.914810  3402 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:40.914872  3591 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.916360  3511 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.920028  3160 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.920778  3160 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:40.927042 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:40.928391 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 98977us
W20260709 12:25:40.929461 31059 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2334) took 1686 ms (client timeout 1801 ms). Trace:
I20260709 12:25:40.930754  3181 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:40.931300  3181 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:40.932794  3181 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.936815  3181 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:40.917522  3511 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:40.939129 31109 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.731s	user 0.022s	sys 0.054s
W20260709 12:25:40.930162 31059 rpcz_store.cc:269] 0709 12:25:39.242683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.390349 (+147666us) service_pool.cc:224] Handling call
0709 12:25:40.928929 (+1538580us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003971d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:40.929157 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2348685184}
I20260709 12:25:40.940379  3511 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.941488 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:40.966385 31058 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:40.943367  3160 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.936991  3250 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.968262  3432 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.968791  3432 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:40.969472  3250 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.946321 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:40.946941 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57890.9us
I20260709 12:25:40.950646  3201 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:40.954232  3583 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.956003 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:40.962632  3372 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:40.944397 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:40.971184  3086 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:40.972728 31109 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.765s	user 0.022s	sys 0.055s
I20260709 12:25:40.972205  3432 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.974309  3335 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.974499  3583 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.975114  3335 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.975804  3512 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.976297  3512 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:40.976446  3491 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:40.972967  3086 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:40.977108  3583 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:40.973364 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 1.76539e+06us
I20260709 12:25:40.971426  3372 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:40.977639  3335 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:40.983125  3559 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:40.983560  3559 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:40.970255  3591 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.978406  3432 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.989277  3335 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:40.989842 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:40.990617  3581 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:40.990828 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:40.990134 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.991384  3581 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.993387  3086 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:40.992851  3559 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:40.995543  3581 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:40.997195 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.998252  3281 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:40.998580  3559 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:40.999076 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 872) took 1581 ms. Trace:
I20260709 12:25:40.999534 30987 rpcz_store.cc:276] 0709 12:25:39.417550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.461665 (+ 44115us) service_pool.cc:224] Handling call
0709 12:25:39.491123 (+ 29458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.539796 (+ 48673us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:39.583907 (+ 44111us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.082276 (+498369us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.082347 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:40.082432 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.998567 (+916135us) spinlock_profiling.cc:243] Waited 48.5 ms on lock 0x7b54001f6b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:40.998633 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.998836 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28841,"prepare.queue_time_us":44385,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":107841,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":19406,"spinlock_wait_cycles":111444224,"thread_start_us":469091,"threads_started":1,"wal-append.queue_time_us":785996}
I20260709 12:25:41.007236 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2341) took 1762 ms. Trace:
I20260709 12:25:41.007663 31065 rpcz_store.cc:276] 0709 12:25:39.244670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.867501 (+622831us) service_pool.cc:224] Handling call
0709 12:25:39.867850 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.084230 (+216380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.006716 (+922486us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003962d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:41.007040 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495931136}
I20260709 12:25:41.010807 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.008731  3372 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.014025  3485 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:41.014681 31141 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:41.014771  3372 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:41.021162  3411 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.022715  3287 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.025518 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.027034 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:41.027191 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 81142.6us
I20260709 12:25:41.027679 31697 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:41.027912 31698 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:41.023434  3287 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.029778  3354 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:41.032999  3287 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:41.033850 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.034814 31064 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:41.036139 31059 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:41.037036 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.029965 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.039355 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.039995 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.040095 31690 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:41.043265 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.043994 31690 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:41.049299  3575 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:41.053766  2970 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:41.054247  2970 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.055670  2970 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.057090  2970 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:41.057260 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.059222 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 115004us
I20260709 12:25:41.059928  3459 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.060993 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.068009 31141 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:41.069876 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.070181 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.071224 31698 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:41.072338 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.073467 31144 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 13: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:41.073881  3588 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:41.075034  3581 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:41.075477  3581 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:41.076659 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.077384 31143 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:41.080096  3585 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:41.080828 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.081656 31141 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:41.083011 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.083243 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:41.083784 31138 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:41.084095 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.078840  3281 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:41.084992 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.095417 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.096910  3560 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:41.098393 31060 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:41.103075 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73339.1us
I20260709 12:25:41.104322  3560 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:41.105847  3560 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:41.106561  3379 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:41.107023  3379 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.107336 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.107551  3452 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:41.108036  3452 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.108402 31143 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:25:41.108148  3379 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.109071  3452 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.110327  3379 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.110195 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:41.111646 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:41.095258  3585 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:41.114615 31140 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 10.
I20260709 12:25:41.116276 31144 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:41.114130  3439 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:41.118343  3439 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:41.118755  3439 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:41.121922 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:41.126204  3557 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:41.126899 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.105875 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:41.129112  3485 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.133489  3283 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:25:41.139372 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.132791  3401 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:41.130250  3250 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:41.096097 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:41.133599  3581 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.139268  3287 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:41.140161  3283 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:41.170418  3283 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:41.170634 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:41.171463  3283 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.172807  3283 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:41.173013  3439 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.131130  3381 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:41.175320  3381 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:41.175742  3381 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:41.176868  3381 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.126768  3557 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.142695  3288 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:25:41.178381 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.143036 31690 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:41.179107  3575 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:41.179580  3581 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:41.179688  3575 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.180294  3575 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.144737  3567 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:41.144873  3452 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:41.150954 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 67004.3us
I20260709 12:25:41.182482  3567 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:41.178310  3557 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:41.134064 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 48466us
I20260709 12:25:41.182951  3567 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.184614 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:41.185410  3567 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.186944  3567 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.187054  3288 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:41.187518  3372 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:41.155660  3524 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:25:41.185724 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:41.178433 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 86703.6us
I20260709 12:25:41.187880  3439 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.160916  3401 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.188670  3287 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.220430 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:41.218250 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.194655  3288 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.194655  3372 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.213696 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.218642  3381 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:41.218899  3287 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:41.230780 31061 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:41.211527  3401 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.235611 31058 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:41.212091  3459 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:41.237545 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54628.3us
I20260709 12:25:41.256418 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.233075  3372 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:41.269620 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 130336us
I20260709 12:25:41.256240 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:41.251669 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:41.236724  3485 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.234589  3354 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:41.247499  3459 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.270040  3432 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:41.279337 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 56051.3us
W20260709 12:25:41.286895 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57068us
I20260709 12:25:41.289155 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:41.266724  3513 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.293232 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.294021 31698 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:41.294445 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.292394 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:41.279814  3459 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.283625  3432 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.297966 31068 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:41.298831 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:41.297333 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.300454 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:41.296556  3533 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:41.301630  3533 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.293315  3411 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:41.302264 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 45969.2us
I20260709 12:25:41.302455  3411 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:41.305222  3411 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.305624 31109 leader_election.cc:400] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:41.305958 31109 leader_election.cc:403] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:41.306545  3086 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:41.308250 31064 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:25:41.302014  3533 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.308921  3201 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:41.296645 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:41.310108  3373 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:41.310386 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.310597  3373 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.311055  3373 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:41.311232  3086 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.312803 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:41.313294  3439 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:41.313704  3439 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.314090  3439 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:41.314796  3563 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.273s	user 0.000s	sys 0.014s
I20260709 12:25:41.316298 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 699) took 1811 ms. Trace:
I20260709 12:25:41.316847  3086 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:41.317534  3086 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:41.318040  3086 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.319029 31065 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:41.298342 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.311081 31059 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:41.316658 31695 rpcz_store.cc:276] 0709 12:25:39.504628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.505264 (+   636us) service_pool.cc:224] Handling call
0709 12:25:39.563210 (+ 57946us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.185455 (+622245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.185543 (+    88us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:41.051672 (+866129us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b2c000b6068. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:25:41.052263 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.191274 (+139011us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:41.202900 (+ 11626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.210104 (+  7204us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.315885 (+105781us) raft_consensus.cc:1685] finished
0709 12:25:41.315903 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.316118 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32958,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":4407,"spinlock_wait_cycles":15802240,"thread_start_us":725720,"threads_started":2,"wal-append.queue_time_us":167320}
I20260709 12:25:41.327759 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2351) took 1823 ms. Trace:
I20260709 12:25:41.299978  3432 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:41.330700  2989 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.329160 31139 rpcz_store.cc:276] 0709 12:25:39.500341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.500853 (+   512us) service_pool.cc:224] Handling call
0709 12:25:39.501110 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.509062 (+  7952us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:39.509090 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.541201 (+ 32111us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.840365 (+299164us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:39.840404 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.840481 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.234113 (+1393632us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:41.319137 (+ 85024us) raft_consensus.cc:1685] finished
0709 12:25:41.319157 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.323974 (+  4817us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7347,"raft.queue_time_us":8043,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":36989,"spinlock_wait_cycles":7900928,"thread_start_us":249445,"threads_started":1,"wal-append.queue_time_us":454097}
I20260709 12:25:41.331122  3580 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:41.334360  3349 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:41.336719  3575 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.338377  3290 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:41.339440  3557 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.342310 31061 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:41.344247  3439 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:41.344265 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.352384 30955 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:41.348181 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:41.352008 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.351274  3411 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:41.355938 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.348127  3323 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:25:41.348670  3524 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:41.353295  3591 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:41.364177 31142 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:41.364029  3591 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.366075  3250 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:41.366542  3250 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:41.367092  3250 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.356346  3201 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.342310  3513 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.368753  3201 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.371071  3575 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:41.349989  3349 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.371568  3557 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:41.371918  3349 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.351234  3487 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:25:41.374019  3487 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:41.374431  3487 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:41.375385  3487 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.376408 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2314) took 1874 ms. Trace:
I20260709 12:25:41.377032  3288 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.378768 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.380224 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:41.381395 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:41.382067 30990 raft_consensus.cc:2368] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14.
I20260709 12:25:41.383020 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:41.393813 31067 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:41.376797 30992 rpcz_store.cc:276] 0709 12:25:39.501826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.501948 (+   122us) service_pool.cc:224] Handling call
0709 12:25:39.542176 (+ 40228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.882825 (+1340649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.376148 (+493323us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54003467d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:41.376354 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2962718080}
W20260709 12:25:41.396431 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 100153us
W20260709 12:25:41.397190 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47983.1us
I20260709 12:25:41.395406  3487 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.400327  3512 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.400619  3160 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:41.401887  3512 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:41.402649 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.402556 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:41.403426 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.404490  3411 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:41.397010 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:41.412554 31139 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:41.412707 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:41.413486 31695 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:25:41.415885 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:41.417052 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.413877 31695 pending_rounds.cc:85] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 9
I20260709 12:25:41.419214 31060 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:41.419915 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.422241 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:41.422191 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.413834  3430 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:25:41.426028  3580 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:41.427274  3459 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.428884 30989 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:41.418349 31695 pending_rounds.cc:107] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:25:41.424463 31139 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:41.435343  3288 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.440420  3086 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.433706 31695 raft_consensus.cc:2889] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:41.445708 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.450016  3464 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:41.452113 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.452864 31139 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.455497  3513 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:41.418102  3575 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.457186  3575 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.458313 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.459060 31068 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:41.460685 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:41.461654 30984 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:41.462855  3580 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:41.463285  3580 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.463598  3580 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.464883  3373 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.466241  3591 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.466714 31136 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:41.467418 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:41.422493  3323 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:41.470402  3381 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:41.470840  3381 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:41.471454  3381 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.472345  3373 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.484228 31143 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:41.484853  3287 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.486968  3287 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:41.489818 31063 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:41.492658 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:51946 (request call id 2340) took 2248 ms. Trace:
I20260709 12:25:41.475400 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.493005 31063 rpcz_store.cc:276] 0709 12:25:39.244424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.850950 (+606526us) service_pool.cc:224] Handling call
0709 12:25:41.402620 (+1551670us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b54003962d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88ba3d 00007f2ef085bcd0 00007f2ee9de0254 00007f2ee9de01e2 00007f2ee9de0111 00007f2ee9de008c 00007f2ee9ddf342 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:41.489105 (+ 86485us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b4c002b2600. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee9b60a20 00007f2ee9b71a29 00007f2ee9b7213f 00007f2ee9b362ab 00007f2ee9b63f3c 00007f2ee9b643fe 00007f2ee9df188b 00007f2eeb7f0495 00007f2eeb84130f 00007f2eeb88ba55 00007f2ef085bcd0
0709 12:25:41.492546 (+  3441us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162182656}
I20260709 12:25:41.477862  3591 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.479061 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:41.498651 30989 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:41.499827  3201 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.502056  3439 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.502607  3591 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.470985  3575 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:41.482977 31140 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:41.483213 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:41.480434 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.476941  3432 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.491632 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.509536 31143 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:41.491459  3459 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.505638 31065 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.490970 31144 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:41.429720  3372 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:41.492079 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.533444 31139 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:41.541460 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:41.542135 31692 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:41.543431 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 121283us
I20260709 12:25:41.497792  3401 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.497725  3381 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.547080 31062 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:41.549718  3486 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:41.530503  3452 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:41.555744 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75391.4us
I20260709 12:25:41.499862  3533 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.554819  3086 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.556805  3290 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:41.571542 31059 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:41.575289 31068 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:25:41.576756 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.576565 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.582513 31060 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:41.599634 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:41.601378 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.601769 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:41.600661 30989 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:41.615368 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.618160 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.621135 31059 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:41.609833  3287 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:41.597921  3354 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:25:41.612573  3567 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:41.604612  3201 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.642524 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.646855 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:41.648880 31695 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:41.653529 31060 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
W20260709 12:25:41.659044 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 43755.6us
I20260709 12:25:41.660929 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:41.658036 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:41.667900 31664 leader_election.cc:400] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14.
I20260709 12:25:41.709764 31664 leader_election.cc:403] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:41.614303  3358 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:25:41.710953 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 50097.1us
I20260709 12:25:41.662475 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:41.717092 31063 raft_consensus.cc:2368] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15.
I20260709 12:25:41.717440  3379 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:25:41.717592 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 57757.4us
I20260709 12:25:41.720142 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.713968 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.721861 31144 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:41.639199  3567 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.647976  3591 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:41.728628 30955 leader_election.cc:400] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15.
I20260709 12:25:41.728953 30955 leader_election.cc:403] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:25:41.711742  3581 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.000s
I20260709 12:25:41.663673  3432 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.647684  3287 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.709992  2989 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:41.739114 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.728374  3567 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.718866  3464 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:41.741290 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:41.746205 31690 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
W20260709 12:25:41.747524 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 105142us
I20260709 12:25:41.723315  3372 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.739439  3401 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:41.743144  3533 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:41.748616  2860 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:41.746009  3381 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:41.748840  3560 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:41.747355  3181 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.765480 31139 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:41.786108 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.819502 31136 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:41.796777 31065 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:41.804472 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.803328 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:41.817775 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 53454.6us
I20260709 12:25:41.826494 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:41.827651 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 109284us
I20260709 12:25:41.763219 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.804776 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.829409 31144 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.829903 31138 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:41.829653 31692 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:41.782065 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.833660 31140 raft_consensus.cc:2376] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14. Re-sending same reply.
I20260709 12:25:41.805081 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.840329 31137 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:41.841847 31135 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:25:41.765987  3612 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:25:41.764492  3255 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.195s
I20260709 12:25:41.846524  3372 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:25:41.846660 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:41.839646 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:41.797789  3160 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:41.892627 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.764770  3577 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:41.803512  3287 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.807703  3250 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.855738 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:41.856460 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.835018  3381 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:41.859736  3379 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14.
I20260709 12:25:41.903901 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:41.860651 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 147239us
I20260709 12:25:41.861369  3560 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:41.778528 30887 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 11, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 11 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:41.866351 31143 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:41.906138 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.867620 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:41.764545  3442 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:41.788275 30884 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 199) took 38895 ms (client timeout 15000 ms). Trace:
I20260709 12:25:41.871717  3580 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:41.832271  3201 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:25:41.878944 31692 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:41.854600  2860 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:41.766393  3452 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:41.900542  3373 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:25:41.920737 31696 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:41.762727  3358 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.891917  3464 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:41.900084 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:41.901003 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.902596 31139 raft_consensus.cc:2410] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:25:41.905356 30984 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:41.931721 30990 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:41.904558 31695 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:41.922675 31141 raft_consensus.cc:2368] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12.
I20260709 12:25:41.927064 31694 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:41.910092 31136 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:41.910794 31144 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:41.922273 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:41.894701  3577 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:41.918740 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.919601  3572 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:25:41.797626  3349 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.929627  3381 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:41.922025 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:41.946542 31691 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:41.936491 30887 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:41.921962 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:41.923868  3580 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:41.913178  3283 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:41.927143  3557 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:41.921574  2860 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:41.963435 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:41.963440 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:41.921913  3452 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.914407  3442 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.942811  3323 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:25:41.963407  3487 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.001s
I20260709 12:25:41.939802  3577 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.949669 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:41.947722  3358 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.947748  3464 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:41.988384 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54578.6us
I20260709 12:25:41.921561  3160 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.991348  3452 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:41.992393  3160 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:41.992723  3452 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.992977 30887 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 10, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 10 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:41.972750  2860 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.989892 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:41.941135  3486 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:41.969827  3557 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:41.983848  3430 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.000722 30887 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 10, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 10 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:42.002938  3250 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.003731 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:42.003502 30887 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 14, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 14 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.005195  2860 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.006433  3160 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:41.987893  3381 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.007575  3358 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:42.008272  3181 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15.
I20260709 12:25:42.009171 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:42.009354 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:42.009812 31061 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:42.011102 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:42.006974  3160 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:42.005215 30887 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 10, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 10 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:41.979024  3283 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.018426 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:41.980466  3323 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:42.033891 31062 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.034132 31139 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:25:42.035709 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 91812.7us
W20260709 12:25:42.047767 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 110463us
W20260709 12:25:42.048202 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58382.8us
W20260709 12:25:42.050263 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 55067.1us
I20260709 12:25:42.052963 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:42.055802 31143 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:42.056228 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:42.057276  3572 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:42.057758  3572 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:42.058501  3373 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:42.059355 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:42.060616  3533 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:42.060730 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:42.061172  3533 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:42.062130  3533 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.063311  3512 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:42.063975  3512 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:42.061939  3572 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.059433  2989 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:42.081082 31690 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:42.015372  3430 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.054780  3086 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.086480 30887 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 11, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 11 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:42.042080  3358 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
W20260709 12:25:42.120998 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.124431 31143 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:42.095273  3287 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:42.065230  3486 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:25:42.133852 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:42.145051 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:42.131227 30887 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.150270 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:42.154438 31143 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:42.112917  3401 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:42.132875  3287 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.168390 31140 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:42.161331 30887 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 11, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 11 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.109063  3512 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.101003  3557 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.167331  3287 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.186265 31143 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:42.187721 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:42.191900  3625 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:25:42.187093 30887 catalog_manager.cc:5697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 8, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 8 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:42.167500  3401 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.156778  3567 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.178787  3349 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.216344 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:42.175498  3591 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:42.113247  3464 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.181298  3160 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.192698  3627 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:42.128716  3323 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.224398 31691 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:42.228925 31062 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
W20260709 12:25:42.229696  3629 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:25:42.231932  3430 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:42.232486  3283 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.232959  3442 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.281935 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:42.282433 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:42.234721 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:42.236222  3401 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:42.236788  3486 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:42.237347 31697 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:42.237650  3372 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.238260  3567 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:42.286999 31138 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:25:42.239018  3637 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:25:42.240554  3631 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:25:42.245893 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:42.246631 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:42.247350 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:42.247538 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:41.908592 30884 rpcz_store.cc:269] 0709 12:25:02.892247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.892559 (+   312us) service_pool.cc:224] Handling call
0709 12:25:41.787983 (+38895424us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:30.589452 (+     0us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2ee878a0ac 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef2c916c
0709 12:25:30.788499 (+199047us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:30.789208 (+   709us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:30.789260 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:25:30.789295 (+    35us) tablet.cc:662] Decoding operations
0709 12:25:30.833639 (+ 44344us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:30.846354 (+ 12715us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:30.848032 (+  1678us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:25:30.923440 (+ 75408us) tablet.cc:701] Row locks acquired
0709 12:25:30.923498 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:25:30.923682 (+   184us) write_op.cc:270] Start()
0709 12:25:30.923966 (+   284us) write_op.cc:276] Timestamp: P: 1783599930923613 usec, L: 0
0709 12:25:30.924002 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:25:30.924733 (+   731us) log.cc:844] Serialized 3954 byte log entry
0709 12:25:31.848700 (+923967us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:32.215850 (+367150us) op_driver.cc:464] REPLICATION: finished
0709 12:25:32.438045 (+222195us) write_op.cc:301] APPLY: starting
0709 12:25:32.438144 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:32.800905 (+362761us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:32.840131 (+ 39226us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:33.009321 (+169190us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:33.021195 (+ 11874us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 12:25:33.021270 (+    75us) write_op.cc:312] APPLY: finished
0709 12:25:33.021768 (+   498us) log.cc:844] Serialized 97 byte log entry
0709 12:25:33.024169 (+  2401us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:33.024357 (+   188us) write_op.cc:454] Released schema lock
0709 12:25:33.024444 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:33.025046 (+   602us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:33.025088 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2265098,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":201744,"apply.run_cpu_time_us":18390,"apply.run_wall_time_us":586950,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":235812,"prepare.run_cpu_time_us":33199,"prepare.run_wall_time_us":470661,"raft.queue_time_us":336790,"raft.run_cpu_time_us":7010,"raft.run_wall_time_us":110823,"replication_time_us":1291652,"spinlock_wait_cycles":83170432,"thread_start_us":732852,"threads_started":4,"wal-append.queue_time_us":645983,"wal-append.run_cpu_time_us":27939,"wal-append.run_wall_time_us":2483616}]]}
I20260709 12:25:42.248450 30887 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 12, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 12 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:42.259636 31063 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:25:42.238761  3630 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:25:42.291743  3627 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:42.296053  3638 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.297503 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:42.283684  3452 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.280054  3640 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.301543 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:42.306818 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:42.299304 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 113529us
I20260709 12:25:42.311717 31693 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:42.313712 31059 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:42.308936 31691 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:42.301275  3486 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:42.269984  3250 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:25:42.270011  3591 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.312785 30887 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.320336 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:42.322329 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:42.322686  3645 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:42.323191  3645 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:42.323411  3636 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:42.323596  3645 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.316850  3638 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.324841 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:42.326572  3638 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:42.315690  3452 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.329411  3452 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:42.301425  3372 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.362541 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:42.360049 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:42.361791 30887 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.349933  3401 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.378376  3401 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.377298  3459 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:42.381053  3459 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:42.381960  3459 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.382967  3459 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:42.331138 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.331949 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:42.361666  3452 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:42.391690  3452 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.352201 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:42.394182 30992 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:42.394954 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:42.395306 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:42.395615 31061 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:42.396381  3430 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.396864  3430 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.397545 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:42.403451 31693 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:42.408257 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:42.398396  3524 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.424690 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:42.409653 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 49727.3us
I20260709 12:25:42.398495  3640 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:42.317754  3533 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:42.456558  3638 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:42.476037  3524 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.392761  2860 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:42.460964  2989 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.349996  3442 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.360265  3250 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:42.422943  3486 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:42.548538 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.565445  3452 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:42.461892 30887 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 13, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 13 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:42.569958  3283 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.562966 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 99934us
I20260709 12:25:42.576061  3464 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:42.427803  3634 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:42.458549  3627 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.578924 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:42.497603  3255 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:42.463725 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:42.617396 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 153758us
I20260709 12:25:42.579278  3581 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:42.330166  3577 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.578133  3201 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.581480  3639 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.428840 30955 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:42.605198 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 283224us
I20260709 12:25:42.560142  3372 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:42.591501  3533 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:42.596120  3640 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:42.630446 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:42.598521  3086 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.598359  3430 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.606112  3181 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.618974  3255 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.628513 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:42.626332 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 202545us
I20260709 12:25:42.628422 30887 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.632691 31061 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:42.612025  3634 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.621299  3638 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:42.667058 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:42.671681  3646 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:25:42.630800  3381 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:42.674615  3381 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:42.675549  3381 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.678350 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:42.679008 30991 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:42.598982  2989 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.682392 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:42.630699  3354 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:42.610162  3283 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.699337 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:42.617647  3464 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.706429  3643 raft_consensus.cc:2764] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:25:42.706542  3591 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.706713  3640 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.669924  3634 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:42.708984 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 95815.3us
I20260709 12:25:42.671279  3255 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:42.710731 30887 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 10, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 10 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:42.628129  3201 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.695432  3401 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.704684  3354 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.721113 30887 catalog_manager.cc:5697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 10, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 10 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:25:42.683115  3512 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.634371  3581 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:42.695600  3588 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.669557  3181 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.796759 30887 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 9, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 9 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:25:42.798243 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.632229  3639 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.793870  3512 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.802136 30992 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:42.833667 30887 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 10, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:25:42.802994  3655 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
W20260709 12:25:42.804077  3652 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 12:25:42.804418 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.806593 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:42.809214  3181 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.809849  2989 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:42.810315  3533 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:42.815263 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 132990us
I20260709 12:25:42.819120  3464 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.852437 30986 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:42.853432 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:42.854460 30983 pending_rounds.cc:85] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 6
I20260709 12:25:42.854784 30983 pending_rounds.cc:107] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:25:42.855001 30983 raft_consensus.cc:2889] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.637704  3086 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.924547 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:42.833431 30991 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:42.794596  3490 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.932497  3490 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.934019 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:42.934600  3490 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.935196 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:42.935822 30992 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:42.935648 30887 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 11, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 11 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:25:42.937800  3287 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.731s	user 0.007s	sys 0.004s
I20260709 12:25:42.937295 30887 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 11, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 11 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.938256  3287 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.939083 30887 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 8, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 8 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.941110  3634 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.840286  3290 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.924466  3255 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.944705  3290 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.945775 30989 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:42.843912  3577 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:42.799705  3588 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.723222  3358 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 12:25:42.925505 31060 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.941885  3581 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:42.942545 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:42.840049  3645 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.004s
I20260709 12:25:42.973203  3645 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:42.975850  3645 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:42.944082  3086 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:42.940682 30887 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:42.997641 30887 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 11, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 11 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:43.000079 30887 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 11, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 11 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:43.041218  3354 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.043128  3201 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.058832  3287 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.058918  3639 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:43.088927 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 386155us
I20260709 12:25:43.059645  3512 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:43.063637 31060 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
W20260709 12:25:43.064049 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 212783us
I20260709 12:25:43.066933  3401 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.067046 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:43.067380  2860 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.067947 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:43.094818 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.078090 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.096035 31691 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:43.078374 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.098253 31690 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:43.078760  3379 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:43.103267 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:43.081956 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.104403 31063 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:25:43.085809 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.105718  3086 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:43.106030 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:43.089429  3577 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.106748 31137 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 10: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:43.106927  3577 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.107357 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.096555  3401 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.108525 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:43.108919  3577 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:43.109114 31135 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.088244  3181 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.109301  3401 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.109738  3181 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.111958  3181 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.115509 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:43.116267 31141 raft_consensus.cc:2368] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15.
I20260709 12:25:43.116344 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.116971 31137 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:43.117113 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.117978 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.118566 31143 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:43.118618 30992 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:43.087198 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.094900 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:43.119736 31065 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:43.077013 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.086819  3381 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.100455 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:43.117758 31141 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:43.080546  3588 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:43.129551 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.094503 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:43.124743  3401 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:43.130492 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.130594  3401 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.131320 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:43.130492 31697 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:43.132059 31694 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:43.132383  3577 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:43.132849  3577 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:43.133698  3577 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.091420  3290 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.129474 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.101878 31062 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:43.096397  2860 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.123868 31135 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:43.129637 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.099131 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:43.137507 31690 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:43.129918 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:43.138331 31140 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:25:43.137398 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.141906 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.142725 31067 raft_consensus.cc:2410] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:25:43.143981 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44965us
I20260709 12:25:43.146497 31691 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:25:43.148168 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 47821.5us
I20260709 12:25:43.148725 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:43.149639 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.150484 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.151180 31692 raft_consensus.cc:2368] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12.
I20260709 12:25:43.126482  3627 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.143402  3588 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:43.147262  3442 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:43.147432  3290 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:43.135043  3381 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.196503 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.197157 31692 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
W20260709 12:25:43.157492 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.842s (SENT) [suppressed 21 similar messages]
W20260709 12:25:43.200029 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.200964 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.201571 31691 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:43.156032  3634 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:43.202992 31693 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:43.203258 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.203900 30986 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:43.204492 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.205098 31696 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:43.205649 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.206259 30988 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:43.206472 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.206697 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.207058 31692 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:43.207129 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:43.207798 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:43.210448 30986 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:43.211207  3442 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:43.211700 30986 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:43.200584  3588 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.212584  3588 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:43.217195  3650 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.794s	user 0.009s	sys 0.015s
I20260709 12:25:43.218425 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2385) took 1172 ms. Trace:
I20260709 12:25:43.221089 31065 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:43.218758 31136 rpcz_store.cc:276] 0709 12:25:42.046027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.046136 (+   109us) service_pool.cc:224] Handling call
0709 12:25:42.078554 (+ 32418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.162005 (+ 83451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.162040 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.177236 (+ 15196us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.301957 (+124721us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:42.302011 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.302111 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:43.216627 (+914516us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:43.218104 (+  1477us) raft_consensus.cc:1685] finished
0709 12:25:43.218129 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.218354 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20364,"raft.queue_time_us":44111,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":3687,"spinlock_wait_cycles":32896256,"thread_start_us":182079,"threads_started":2,"wal-append.queue_time_us":131709}
I20260709 12:25:43.224633 30988 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:43.156704  3627 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:43.228314 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 79413.7us
I20260709 12:25:43.200223 31690 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:43.231278 30989 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
W20260709 12:25:43.241608 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 138441us
I20260709 12:25:43.246129 31697 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:43.249893 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:43.232307  3401 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:43.252383  3201 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:43.253201  3373 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:43.247761  3645 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:25:43.284791 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:43.262003  2860 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:43.286718 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57796.7us
I20260709 12:25:43.287482 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:43.288785 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:43.292734  3250 raft_consensus.cc:2764] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:43.298799  3442 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:43.307693  3666 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:25:43.311295  3255 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:43.312228  3323 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:43.305480  3588 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.312912  3627 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:43.314546  3668 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:25:43.315806 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:43.316416 31061 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:43.316583  3381 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:43.322625 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 59845.3us
I20260709 12:25:43.323678 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:43.323793 31061 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:25:43.324899  3672 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:43.325359 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:43.325728  3672 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:43.326558  3669 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:43.327219  3669 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:43.327479 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:43.328193  3669 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.331086 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.331696 31144 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:43.332687 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.332924 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.333351 31136 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:43.333534 31139 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:43.330513 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:43.335593 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:43.351471 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.353034 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:43.353729 31065 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:43.356261 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:43.356815 30985 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:43.361482  3627 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:43.361665  3283 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.361910  3627 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.362325  3627 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.352334 31068 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:43.367219 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:43.368119  3557 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:43.368498 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:43.369279  3640 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:43.369741 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:43.370106  3643 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:43.370600 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 901) took 1155 ms. Trace:
I20260709 12:25:43.372457 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.382272 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.382740 31063 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:43.382974 31058 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:43.373090 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.383662 31060 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:43.384043 30990 rpcz_store.cc:276] 0709 12:25:42.215177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.215335 (+   158us) service_pool.cc:224] Handling call
0709 12:25:42.215657 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.216011 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.216044 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.216516 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.515727 (+299211us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:42.515785 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.515879 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:43.357249 (+841370us) spinlock_profiling.cc:243] Waited 7.65 ms on lock 0x7b2400175f58. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee862d9c4 00007f2ee862bacb 00007f2ee87e6dc9 00007f2ee87e6a55 00007f2ee87e5fb0 00007f2eeb45c488 00007f2eeb45cbe8 00007f2eeb434398 00007f2eeb42f76a 00007f2eeb42ebb6
0709 12:25:43.370286 (+ 13037us) raft_consensus.cc:1685] finished
0709 12:25:43.370307 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.370540 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":893,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":10057,"spinlock_wait_cycles":17594752,"thread_start_us":298720,"threads_started":1,"wal-append.queue_time_us":258894}
I20260709 12:25:43.369793  3640 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:43.374531  2989 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.386507  2989 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.386824  3640 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.389128  2989 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:43.374977  3372 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.376951 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.390055 31062 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:43.389927  3372 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.390779 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:43.391527 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56032.1us
I20260709 12:25:43.392290  3634 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:43.392501  3372 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:43.392627 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:43.392935  2860 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:43.393316  2860 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.393648  2860 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:43.394374  3665 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:43.330663  3373 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:43.396550  3373 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.342885  3160 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:43.398455  3160 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.380244 30985 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:43.400895 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.401268 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:43.402374 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:43.403324 31065 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:43.400399 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.401567 31068 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:43.314850  3524 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.407382 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:43.404852 31067 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:43.333526  3588 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:43.411722 31063 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:25:43.415979 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.352507  3358 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:43.421360 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 51689.5us
I20260709 12:25:43.392683  3634 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.416932 30955 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:43.406173  3581 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:43.415681  3290 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:43.412437  3354 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:43.408879  3250 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:43.408667  3287 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.432179 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:43.456794 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:43.428759  3634 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:43.431353  3283 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:43.428979  3358 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:43.437330  3354 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.440371  3290 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:43.475975 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:43.484166 30955 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:43.460386  3381 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:43.469837  3160 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.471423  3283 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.467943  3379 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:25:43.493891 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 51791.4us
W20260709 12:25:43.496920 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 208223us
I20260709 12:25:43.508332 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:43.474342  3672 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:43.473181  3560 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.474922  3354 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:43.511097 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:43.491497  3663 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:43.475360  3250 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.513029 31058 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:43.507321  3283 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:43.492009  3381 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.468497  3287 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.495618  3487 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:43.515759 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.498018  3430 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:25:43.534538 31142 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:25:43.540648  3290 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.548889 31135 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:43.513142  3524 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:43.525754  3663 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.552242 30958 leader_election.cc:400] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12.
I20260709 12:25:43.554201 30958 leader_election.cc:403] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:25:43.549175  3381 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.510672  3379 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.550871  3487 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.552933  3255 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:43.563527  3487 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:43.606762  3677 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:25:43.593279 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:43.591573  3287 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.592916  3379 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:43.609916  3678 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:25:43.526198  3560 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.582616 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.612700 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.598958 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:43.619297 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.619874 31136 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:43.598464 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.602012 30986 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:43.621055 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.564567 31140 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:43.601195 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:43.619976 30989 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:43.566123 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.621935 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:43.621896 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 213525us
I20260709 12:25:43.625536 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.627718 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.628314 31063 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:43.599391 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.555284  3663 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.627319  3679 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:43.634588 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 82492.8us
I20260709 12:25:43.635308 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:43.582759  3676 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:25:43.627300  3401 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.627871 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.630589 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:43.563621 30990 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:43.648294 31058 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:43.652002 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:43.663371 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.663141  3627 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.663952 31697 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:43.671403 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.672200 31135 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:43.658619 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.670205 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.661514 31661 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:43.665599  3283 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.687245 30983 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:43.549571  3358 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.699112  3693 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:43.699620  3693 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.700048  3693 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:43.708276  3693 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.713902 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:43.714452 31058 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:43.721490 31058 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:43.722570 31661 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:43.723383  3694 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:43.725674  3694 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:43.726600  3694 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.732596 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:43.593714  2860 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:43.733363 31697 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 14: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:43.961333 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:43.961714  2860 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:43.945984 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:43.963730 31698 raft_consensus.cc:2410] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 12 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:43.776798 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:43.940383 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.676836  3639 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:43.947121 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:43.947723  3591 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:43.944304  3669 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:43.657025  3401 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:43.797356 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:43.658595  3581 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.658392  3442 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:43.952205 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.029282 31059 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:44.029701 31139 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:43.953598 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:43.766562 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 107454us
I20260709 12:25:44.030716 31063 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:43.735476  3575 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:43.779886 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 82050.1us
I20260709 12:25:43.965524 31135 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:43.961983 31144 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:44.032998 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:44.007607 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:43.956641  3557 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.588s	user 0.016s	sys 0.001s
I20260709 12:25:43.943696  3452 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.023530 31067 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 13. Ops: []
I20260709 12:25:44.025241 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:44.017976  3683 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.210s	user 0.000s	sys 0.008s
I20260709 12:25:44.034302 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.037606 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:44.044667 31690 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 8.
I20260709 12:25:44.044800 30992 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:43.965197  3647 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:44.045562  3647 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:44.047161  3697 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:25:44.048774  3560 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:44.041066 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:44.041399 30987 raft_consensus.cc:2410] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:25:44.052125 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.053150 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:43.971102  3639 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:43.964186  3691 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:25:44.017128  3665 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.065887 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:44.038280  3693 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.068328 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.017716  3591 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.057742  3639 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.058104  3567 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.078617 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:44.054571  3250 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.025452  3669 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.066548  3647 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.034005  3581 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.059082  3452 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.043206  3575 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.100394 31138 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:25:44.094619 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2328) took 1372 ms (client timeout 1636 ms). Trace:
W20260709 12:25:44.130146 30987 rpcz_store.cc:269] 0709 12:25:42.722545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.800790 (+ 78245us) service_pool.cc:224] Handling call
0709 12:25:44.094396 (+1293606us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54003453d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:44.094548 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2740108544}
I20260709 12:25:44.098747 31690 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 13. Ops: []
I20260709 12:25:44.098667 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:44.081826  3679 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12.
W20260709 12:25:44.139343  3634 consensus_meta.cc:287] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.000s
I20260709 12:25:44.143929 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.067147  3665 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.143919 31140 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:44.145759 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.141178 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.141963 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.142038 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.148602 31135 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.148695 30992 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.147143 31139 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
W20260709 12:25:44.095355  3381 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.002s
W20260709 12:25:44.153321 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 120398us
I20260709 12:25:44.091513  3669 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:44.128537  3379 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.156101 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:44.156805 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 125855us
I20260709 12:25:44.089417  3255 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:44.157819 30983 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:44.163801 30958 leader_election.cc:400] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15.
I20260709 12:25:44.168042 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.173522 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:44.178033 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:44.178767 31059 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:44.164638 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:44.088008  3567 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.183388 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:44.184072 31067 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:44.187106 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.182197 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:44.192600 31135 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:44.197306 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.196149 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:44.191080  3688 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:25:44.203445 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:44.167155  3379 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.192992  3442 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:44.210175 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 168083us
I20260709 12:25:44.181519  3691 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:44.154567  3634 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:44.196821  3487 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.003s
I20260709 12:25:44.182329  3283 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.190868  3627 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:44.212172 31068 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:44.160893  3381 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.197088  3669 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:44.200512  3255 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:44.220875  3533 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:44.207544  3672 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:44.197355  3557 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:25:44.210071  3442 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.242926 31693 raft_consensus.cc:2368] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
W20260709 12:25:44.242935 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 86939.2us
I20260709 12:25:44.246891 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.221007 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.271500 31065 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:44.287693 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.288607  3567 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.303630 31139 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:44.313010 31139 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:44.303172 30984 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:44.270776  3255 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:44.319589 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 123517us
W20260709 12:25:44.301702  3685 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:44.238466  3487 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.334942 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.303817  3442 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:44.323520 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:44.328398 31690 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:44.221520  3691 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.290474  3533 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:44.336504 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 93534.8us
I20260709 12:25:44.343487 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:44.301720  3704 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:44.299494  3588 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:25:44.348644 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:44.353132 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:44.350840 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 152915us
I20260709 12:25:44.329685  3255 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.302712  3643 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:25:44.360713 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:44.371738 31058 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.362210 30988 raft_consensus.cc:2368] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:44.370386 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.372684 30984 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.373878  3487 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.373898 30958 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:44.357450  3354 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.822s	user 0.005s	sys 0.005s
I20260709 12:25:44.378489 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:44.351013  3533 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:44.323558  3669 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:25:44.393636 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.394869 30990 raft_consensus.cc:2410] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:25:44.379884 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.396032 30983 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:44.350829  3524 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:44.350457  3704 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.375006  3672 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:44.374495  3581 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:44.399737 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.376466  3701 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:44.402904 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.408423 30984 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:44.391590 31065 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.369627  3591 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:44.393646 30985 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:25:44.422741 31058 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:44.423120 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.428138 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:44.393921  3663 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.761s	user 0.011s	sys 0.000s
I20260709 12:25:44.379730  3703 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:44.439872 30990 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:44.441970 31061 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:44.441808 30983 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.394233  3430 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:44.443846 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.447808 30985 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:44.444638 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:44.447656 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 130218us
I20260709 12:25:44.399662  3381 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.399540  3452 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:44.469213  3691 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.390259  3679 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:25:44.430816  3672 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:44.447427  3663 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.459263 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.416288  3704 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.429296  3669 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:44.487010 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 133940us
W20260709 12:25:44.464291  3718 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:25:44.487550 31059 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:44.488139 31061 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:44.505863 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.466295 30985 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:44.491861 31065 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:44.496098 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.463990  3401 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.496548 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.509310 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.396711  3354 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.499958 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.500017 30992 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:44.503852 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.501463 30986 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:25:44.505077 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:44.508679 31059 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:25:44.506814 30955 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:44.508412 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:44.511198 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:44.516847 30990 raft_consensus.cc:2368] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:44.501437  3672 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:44.525990 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.526587 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:44.526861 30992 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:44.525947 30955 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:44.503821  3634 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.505872  3715 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:44.508615  3669 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:44.542698  3705 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:25:44.543104  3665 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.533846  3627 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:44.544018  3627 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:44.544422  3627 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.544945  3575 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:44.547531  3323 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:44.534265 30990 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:44.502455  3588 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:44.533177  3581 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:44.527683 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:44.535449 31694 raft_consensus.cc:2368] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:44.531612  3452 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:44.507980  3643 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:25:44.553541 31697 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:44.534454  3358 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:44.535640  3701 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:44.551002 31690 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:44.539980 31067 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:44.552703 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:44.559231 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.559989 30990 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:44.560493 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:44.557799 30992 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:25:44.529616  3401 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:44.551594  3715 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:44.566789  3663 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.570080 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:44.570729 31696 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:44.572660 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.558386  3560 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:44.577085 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1115) took 1018 ms. Trace:
I20260709 12:25:44.551213  3323 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:44.574039  3401 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:44.569569  3634 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:44.579494 31067 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:44.577445 31693 rpcz_store.cc:276] 0709 12:25:43.565730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.565860 (+   130us) service_pool.cc:224] Handling call
0709 12:25:44.576897 (+1011037us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400738cd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:44.577034 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1459934464}
I20260709 12:25:44.561704  3452 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:44.578998 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.591732 30990 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:44.590710 30992 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:44.597402 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:44.597797 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.570330  3358 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:44.600052 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.600682 31691 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:44.602478 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.598418 31140 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:44.600068 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:44.596060 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.576120  3560 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.614447  3560 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:44.614706  3401 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.618547 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.618623  3452 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.619283 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.619784 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.620409 31141 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.619853 31140 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:44.588626  3701 consensus_queue.cc:1243] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:44.622144 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.622467 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.623687 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.624420 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.624704 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.625240 31142 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:44.625440 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.625957 31137 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.627982  3290 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:44.628968  3669 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.630906 30987 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:44.631696 30991 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:44.632623 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.633216 31140 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.633338 31696 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:44.633095 31067 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:44.631776 31139 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:44.635171 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.635722 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:44.640735 30985 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:44.641014 30989 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:44.654349 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.654460 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.656213 30987 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:44.656749 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.657764 30990 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:44.613665  3533 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.594377  3721 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:44.663350 31137 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:44.664058 30989 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:44.665831 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:44.601028  3634 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:44.678910 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.679787 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.680200 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.681032 30989 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
W20260709 12:25:44.684625 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48977.5us
I20260709 12:25:44.685603 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:44.651926  3575 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:44.651175  3557 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:44.667173  3640 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.692898 31067 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:44.697492 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.720144 30987 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:44.577095  3442 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.701565 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.726720 30992 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:44.660822  3665 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:25:44.663533  3567 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:25:44.733579 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:44.742751 30984 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:44.744508 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 73971.3us
I20260709 12:25:44.746173 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:44.748270 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:44.678602  3721 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.731499  3354 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:44.759269  3730 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:25:44.759392  3725 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:25:44.695557  3715 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.760689  3290 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:44.761089  3290 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.761390  3290 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.762749 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:44.762969 30983 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:44.763046 31141 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:44.764266 30992 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:44.753660  3287 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:44.767654  3287 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:44.768232  3287 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.723915  3557 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.772559  3557 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.774058  3287 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.778262  3290 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.780825  3290 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.783404 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:44.787204  3533 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:44.743742  3575 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:44.693065  3323 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.791392 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:44.792785  3290 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:44.733709  3701 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:44.793923  3575 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:44.743629  3588 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:44.794387  3575 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:44.795547  3575 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.797101  3575 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.725605  3640 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.798836  3379 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:44.702337 30989 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:25:44.800122  3640 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:44.801920  3627 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.751171  3255 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.793385  3290 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.806195  3255 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:44.806543  3290 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:44.808061  3728 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:44.810990  3442 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:44.812219 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.813017 31141 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
W20260709 12:25:44.814472 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 409736us
I20260709 12:25:44.814946  3627 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.815207 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:44.816956  3721 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:44.818410  3711 raft_consensus.cc:2764] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:44.818642 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:44.819890 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:44.821002  3512 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:44.821825 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.822449 31136 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:44.823311  3639 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.723s	user 0.008s	sys 0.002s
I20260709 12:25:44.823719  3639 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.825239  3250 raft_consensus.cc:2764] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:44.825913  3250 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:44.728130  3634 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.725775  3358 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.826864  3721 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:25:44.827255  3733 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:25:44.829093 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.830772 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:44.858204 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.831192 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.858250 31138 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:44.859586 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.860334 31144 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:44.834287 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.834582 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:44.861244 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.861704 31139 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:25:44.835697  3430 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.836721 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.863046 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.838364 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:44.863726 30987 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.840994 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:44.852921 31140 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:44.871495 30992 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:44.825747  3639 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:44.873359 30958 leader_election.cc:400] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12.
I20260709 12:25:44.873824 30958 leader_election.cc:403] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:44.877585 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 114896us
I20260709 12:25:44.878336 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:44.878680 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.879262 31144 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.879685  3672 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.855667 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:44.881016 31143 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:44.837620 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:44.881697 31137 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.818025  3714 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:25:44.884322  3290 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.880259  3591 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:44.889304  3354 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:44.892856  3704 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:44.895534 30989 raft_consensus.cc:606] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:44.895910 30989 raft_consensus.cc:614] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:44.897377 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:44.898692  3354 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:44.899154  3354 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:44.899714  3354 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.901110 30991 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 13: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:44.907251 31137 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:44.910629 31144 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:44.925879 30987 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:44.927167 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:44.927960 31058 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.935022 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:44.935745 30984 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:44.854074 31136 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
W20260709 12:25:44.941395  3725 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:44.946233 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:44.874469  3381 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.952350 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:44.953634 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 138491us
I20260709 12:25:44.928730  3255 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12.
I20260709 12:25:44.957266 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:44.925639  3290 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:44.961983 31058 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:44.941583  3716 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:44.971284 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:44.900203  3430 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.970600 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:44.969369 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:44.972572 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:44.976055 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:44.950744  3665 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:44.950321  3201 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:44.952472  3287 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:44.985131 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:44.988924  3714 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:44.975384  3716 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:44.996954  3716 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:44.999396 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:45.004961 31698 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:45.008116 30984 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:25:45.009543 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:45.012183 31698 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.974385  3712 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:44.950151  3283 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:25:45.045609 31696 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:45.021592 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:45.023164  3710 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:25:45.014775  3679 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:45.026316 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:44.974882  3734 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:44.987351  3711 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:45.043056 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:45.078271 31144 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.988198  3715 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:44.959481  3486 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:25:45.079977 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:45.069911 31136 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:45.087328 31696 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:45.069327 31061 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:45.071929 31694 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:44.973704  3703 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
W20260709 12:25:45.045681  3730 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:45.083521 31058 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 13. Ops: []
I20260709 12:25:45.095507  3669 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:45.029610  3287 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:45.064169  3250 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:45.066773  3726 raft_consensus.cc:2764] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:25:45.111984 31138 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:45.117762 31144 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:45.071141  3716 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:45.004444  3665 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.172477 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:45.173470 31138 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:45.004805  3201 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.082638  3711 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.079042  3379 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.192479  3379 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.195013  3379 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:45.140684 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 65501.2us
I20260709 12:25:45.015359  3717 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:25:45.180816 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:45.199091  3711 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:45.200568  3737 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:45.201809 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2409) took 1021 ms. Trace:
I20260709 12:25:45.202271 31064 rpcz_store.cc:276] 0709 12:25:44.179984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.184971 (+  4987us) service_pool.cc:224] Handling call
0709 12:25:44.185323 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.247667 (+ 62344us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:44.575345 (+327678us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b180026a100. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8a75e5 00007f2eeb8a7568
0709 12:25:44.638859 (+ 63514us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.811926 (+173067us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b180026a100. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2eef76858f 00007f2ee87a7571 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd
0709 12:25:44.877501 (+ 65575us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.877942 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.878070 (+   128us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:44.878099 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.878156 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.201619 (+323463us) raft_consensus.cc:1685] finished
0709 12:25:45.201641 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.201757 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88239,"prepare.queue_time_us":128535,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":4066,"raft.queue_time_us":110675,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":10437,"spinlock_wait_cycles":61865600,"thread_start_us":282605,"threads_started":3,"wal-append.queue_time_us":279739}
I20260709 12:25:45.201838 30986 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:45.191195  3287 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.189849  3201 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:45.135934  3702 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.077009  3560 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:45.227080 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 200632us
I20260709 12:25:45.284138  3715 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:45.085054  3734 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.148180  3638 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:45.148517  3442 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:45.400121 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 378570us
I20260709 12:25:45.402122 31141 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:45.406906 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:45.407167 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 178703us
I20260709 12:25:45.408295 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:45.447399  3747 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
I20260709 12:25:45.178277  3716 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.454151 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:45.404837  3672 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:45.462289 31033 leader_election.cc:400] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
W20260709 12:25:45.472500 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 65696us
I20260709 12:25:45.473542 31033 leader_election.cc:403] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:25:45.473397 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:45.476629 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:45.429382  3486 raft_consensus.cc:2764] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:25:45.293040  3702 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.457039  3726 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:45.456661  3201 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:25:45.481734  3634 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:45.478637  3283 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:45.486866  3160 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:45.492447  3742 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
W20260709 12:25:45.493637  3743 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:25:45.494508  3739 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.405s	user 0.001s	sys 0.000s
I20260709 12:25:45.496263  3533 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:45.430334  3715 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:45.417124  3638 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:45.497609  3741 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
W20260709 12:25:45.498373  3749 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 12:25:45.499032  3716 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:45.499387 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:45.503050 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:45.500833  3283 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.460294  3712 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:25:45.518422 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:45.521909 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:45.525667  3758 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.530329  3638 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:45.525667  3283 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:45.522799  3726 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:45.540627 30989 raft_consensus.cc:2368] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17.
I20260709 12:25:45.548893 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:45.517009  3430 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:45.537012  3663 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:45.545214  3560 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:45.545665  3702 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:45.547464  3748 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:25:45.547768  3754 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:45.517665  3759 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:45.570752 31064 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:25:45.545255  3751 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:25:45.572273 31698 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.002s
I20260709 12:25:45.522140  3160 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:25:45.576558 31698 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:45.578774 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:45.526002  3691 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:25:45.549165  3533 raft_consensus.cc:2764] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:25:45.598902 31694 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:45.601289 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:45.576737  3665 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:45.590847  3745 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:25:45.587745  3760 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:45.576994  3201 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:25:45.577126  3663 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.716089  3663 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:45.713043 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2589) took 1137 ms. Trace:
I20260709 12:25:45.716975 30985 rpcz_store.cc:276] 0709 12:25:44.575756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.575978 (+   222us) service_pool.cc:224] Handling call
0709 12:25:44.631830 (+ 55852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.641242 (+  9412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.712816 (+1071574us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b54003462d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:45.712976 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127812480}
I20260709 12:25:45.719581  3752 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.721015  3663 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:45.721634  3663 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:45.723850  3663 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:45.730459  3734 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:45.731174 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 175275us
I20260709 12:25:45.609750  3759 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:45.733610  3760 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:45.755966  3742 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:45.754381  3710 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:45.757135  3745 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:25:45.756788  3759 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:45.776383  3533 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:25:45.771674  3714 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:45.788599 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1281) took 1155 ms. Trace:
I20260709 12:25:45.789053 31062 rpcz_store.cc:276] 0709 12:25:44.633060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.634820 (+  1760us) service_pool.cc:224] Handling call
0709 12:25:44.635148 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.831940 (+196792us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:44.838352 (+  6412us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.975096 (+136744us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:45.003123 (+ 28027us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.003302 (+   179us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:45.003347 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.003444 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.788357 (+784913us) raft_consensus.cc:1685] finished
0709 12:25:45.788383 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.788530 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":202433,"prepare.queue_time_us":16717,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":198979,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":20114,"thread_start_us":136331,"threads_started":1,"wal-append.queue_time_us":283826}
I20260709 12:25:45.720280  3752 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.548179  3758 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.808226  3758 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:45.768929  3710 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.795296  3160 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:45.785997  3714 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.767920  3742 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.819723  3742 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:45.800024  3752 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:45.787045  3760 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.839907  3714 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:45.792948  3759 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.815899  3710 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:45.781507  3354 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.833s	user 0.009s	sys 0.000s
W20260709 12:25:45.861443 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 106768us
I20260709 12:25:45.862366 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2225) took 1241 ms. Trace:
I20260709 12:25:45.864300 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:45.816615  3160 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:45.865299 31694 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:45.862861 31135 rpcz_store.cc:276] 0709 12:25:44.620714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.643115 (+ 22401us) service_pool.cc:224] Handling call
0709 12:25:44.663574 (+ 20459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.663776 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:44.743087 (+ 79311us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.861910 (+1118823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.861981 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:45.862050 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.862131 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.862297 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":922097,"prepare.queue_time_us":28719,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":250387,"threads_started":2,"wal-append.queue_time_us":267864}
I20260709 12:25:45.866500 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:45.868382 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:45.868615 31137 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:45.869009 30985 raft_consensus.cc:2376] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15. Re-sending same reply.
I20260709 12:25:45.869895 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:45.872515 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 994217us
I20260709 12:25:45.873437 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:45.874435 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:45.864917  3591 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:45.878273 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:45.878819 30987 raft_consensus.cc:2376] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17. Re-sending same reply.
I20260709 12:25:45.865772  3588 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:45.856179  3354 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.888231 31033 leader_election.cc:400] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17.
I20260709 12:25:45.893261 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:45.893949 31138 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:45.895352 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:45.896277 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:45.901669 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:45.904543 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:45.880298  3591 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:45.914853 31693 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:45.915525 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:45.916002 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:45.916764 31138 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:45.917647 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:45.918604 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:45.902602  3711 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.703s	user 0.004s	sys 0.007s
I20260709 12:25:45.925789 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:45.937649 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:45.938257 31692 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:45.920600  3643 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:45.922385  3710 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:25:45.931036  3714 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:45.931272  3672 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:45.930879  3721 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:45.978003 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:45.979292 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 63856.4us
I20260709 12:25:45.980512 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:45.983470 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:46.000511 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 935) took 1066 ms. Trace:
I20260709 12:25:46.001948 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:45.938169  3742 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:25:46.000906 30983 rpcz_store.cc:276] 0709 12:25:44.933917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.938322 (+  4405us) service_pool.cc:224] Handling call
0709 12:25:44.938703 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.204828 (+266125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.204861 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:45.205329 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.283354 (+ 78025us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:45.283393 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.977724 (+694331us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54001608d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:45.977765 (+    41us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.977804 (+    39us) raft_consensus.cc:1685] finished
0709 12:25:45.977816 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.000448 (+ 22632us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10866,"raft.queue_time_us":3227,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":66791,"spinlock_wait_cycles":611263872,"thread_start_us":77627,"threads_started":1,"wal-append.queue_time_us":285299}
I20260709 12:25:45.920455  3160 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:45.977219  3701 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.010658 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2218) took 1393 ms. Trace:
I20260709 12:25:46.010771 31109 leader_election.cc:400] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:46.013005 31109 leader_election.cc:403] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:25:45.946842  3711 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.012883 31143 rpcz_store.cc:276] 0709 12:25:44.617384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.617501 (+   117us) service_pool.cc:224] Handling call
0709 12:25:44.617770 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.881444 (+263674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.010422 (+1128978us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400063e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:46.010605 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604919040}
I20260709 12:25:46.017181 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:46.017668 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:46.019008 31143 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:46.011576  3721 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:46.021193 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2418) took 1160 ms. Trace:
I20260709 12:25:46.029372 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:46.031034 31063 rpcz_store.cc:276] 0709 12:25:44.860558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.073721 (+213163us) service_pool.cc:224] Handling call
0709 12:25:45.074123 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.195894 (+121771us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.195935 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.195974 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:45.196005 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.020969 (+824964us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000c5b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:46.021012 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.021124 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279564032}
W20260709 12:25:46.034825 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 85712.6us
I20260709 12:25:46.035810 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:46.036801  3788 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:46.037393  3788 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.037868  3788 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:46.038764  3679 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:46.039532  3691 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:46.040534  3721 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:46.040566  3691 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:46.041141  3691 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:46.041905  3781 raft_consensus.cc:2764] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:25:46.042023  3691 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.042697  3781 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:46.043421  3691 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.970279  3591 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:46.063274  3591 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.016498  3283 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:46.040762  3786 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:46.048533 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:46.049186  3774 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:25:46.016009  3701 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.096378 31138 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:25:46.097106  3764 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.502s	user 0.004s	sys 0.002s
I20260709 12:25:46.099313  3758 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:25:46.055768  3778 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:25:46.033466  3512 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:46.101300  3512 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:46.102118  3283 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:46.041107  3721 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:46.103083  3721 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:46.104534  3738 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.101s	user 0.006s	sys 0.009s
I20260709 12:25:46.106655 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 743) took 1095 ms. Trace:
I20260709 12:25:46.108377 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2586) took 1612 ms. Trace:
I20260709 12:25:46.108858 30988 rpcz_store.cc:276] 0709 12:25:44.496055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.496199 (+   144us) service_pool.cc:224] Handling call
0709 12:25:46.108158 (+1611959us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b5400227254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:46.108311 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064478592}
I20260709 12:25:46.110337  3634 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:46.111508  3634 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.114114 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2224) took 1493 ms. Trace:
I20260709 12:25:46.114604 31142 rpcz_store.cc:276] 0709 12:25:44.620424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.625807 (+  5383us) service_pool.cc:224] Handling call
0709 12:25:44.626094 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.652764 (+ 26670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.652810 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.664097 (+ 11287us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.848916 (+184819us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:44.871644 (+ 22728us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.846726 (+975082us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b1800320740. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:25:45.846911 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.847018 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.066753 (+219735us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:46.113870 (+ 47117us) raft_consensus.cc:1685] finished
0709 12:25:46.113898 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.114048 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45257,"prepare.queue_time_us":23990,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":1011203,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":19457,"spinlock_wait_cycles":67555072,"thread_start_us":129826,"threads_started":1,"wal-append.queue_time_us":261001}
I20260709 12:25:46.125010 30988 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:25:46.107000 31690 rpcz_store.cc:276] 0709 12:25:45.011587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.011778 (+   191us) service_pool.cc:224] Handling call
0709 12:25:45.012068 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.250178 (+238110us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:45.434393 (+184215us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.501615 (+ 67222us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.105289 (+603674us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b18001c2b80. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:25:46.105416 (+   127us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.105491 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:46.105558 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.106466 (+   908us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54006ee6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:46.106500 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.106604 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":811,"prepare.queue_time_us":246563,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":745,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":13982,"spinlock_wait_cycles":555862144,"thread_start_us":103118,"threads_started":2,"wal-append.queue_time_us":109079}
W20260709 12:25:46.136272 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 100605us
W20260709 12:25:46.156491 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 132839us
I20260709 12:25:46.157618 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:46.143718  3742 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:46.169160  3742 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.171752  3701 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.172472  3714 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:46.173034  3714 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:46.180184 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75924.7us
I20260709 12:25:46.182919 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.188817  3701 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.203128  3701 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.218828 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:46.219450 31142 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:46.223742 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:46.224893  3753 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:46.224678  3786 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:46.225461  3792 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:46.226691 30992 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:25:46.229013  3703 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:46.181104  3354 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:46.229985  3575 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:46.225458 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:46.232153 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 918) took 1575 ms. Trace:
I20260709 12:25:46.232200  3711 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.233179 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 931) took 1393 ms. Trace:
I20260709 12:25:46.293669 30990 rpcz_store.cc:276] 0709 12:25:44.839494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.842323 (+  2829us) service_pool.cc:224] Handling call
0709 12:25:44.842682 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.934515 (+ 91833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.934560 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.946795 (+ 12235us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.074503 (+127708us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:45.074557 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.232786 (+1158229us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b540029aed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:46.232839 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.232900 (+    61us) raft_consensus.cc:1685] finished
0709 12:25:46.232914 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.233120 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33765,"raft.queue_time_us":40827,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":2384,"spinlock_wait_cycles":132642560,"thread_start_us":93237,"threads_started":1,"wal-append.queue_time_us":229535}
I20260709 12:25:46.238063 31142 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:25:46.293321 30991 rpcz_store.cc:276] 0709 12:25:44.656446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.656548 (+   102us) service_pool.cc:224] Handling call
0709 12:25:46.231963 (+1575415us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540020fb54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:46.232095 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560060160}
W20260709 12:25:46.238842 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.272536 31138 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
W20260709 12:25:46.299489 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 74999.4us
I20260709 12:25:46.300207 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:46.273185 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.303869 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:46.304996 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 143454us
I20260709 12:25:46.311503 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:46.313122 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:46.314256 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:46.314936 30991 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:46.317235 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:46.298534 31109 connection.cc:256] RPC call timeout handler was delayed by -1.2002701039996282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 12:25:46.318028 31109 connection.cc:256] RPC call timeout handler was delayed by -1.1921701039996151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
I20260709 12:25:46.318908 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:46.293493  3703 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:46.337379 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:46.320645 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:46.338521 31695 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:46.343967 30991 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:46.346393 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:46.305752  3575 consensus_queue.cc:1243] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:46.348675 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:46.313273  3639 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:46.305006  3647 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:46.347186  3726 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.778s	user 0.014s	sys 0.008s
I20260709 12:25:46.345973  3760 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.351714  3287 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:46.449090 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.449920 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:46.450659 31138 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:46.292074  3753 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.359222  3575 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.352371  3665 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:25:46.491895 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 143282us
I20260709 12:25:46.498978 31062 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:46.436450  3701 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.430902  3647 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:46.445607  3726 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:46.517297 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:46.451766  3638 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:46.459604  3735 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:46.525175  3796 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:46.499814  3287 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:46.528162  3769 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.528693  3791 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:46.529799 31140 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:46.529867 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.621840 31067 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:46.460402  3703 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.555584 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.625361 31062 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:46.512454  3793 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:46.522897  3647 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.452739  3707 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:25:46.618469 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.521585  3742 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.546339  3639 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:46.550573  3786 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:46.619940 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 301083us
I20260709 12:25:46.610368  3769 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:46.611749  3753 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:46.460999  3760 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:46.524180  3665 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:46.529394  3638 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:46.627019  3250 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:46.640793  3770 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:46.514196  3575 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:46.620018 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.640060 31138 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:46.650013 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:46.670472 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:46.625183  3788 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.587s	user 0.024s	sys 0.018s
W20260709 12:25:46.717371 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 66386.6us
W20260709 12:25:46.675587  3796 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:25:46.679347  3640 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:46.671376  3581 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:46.756944 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 86558.7us
I20260709 12:25:46.722239  3638 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:46.764662 31060 raft_consensus.cc:2410] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:46.758486  3788 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.791837 31109 leader_election.cc:400] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:46.792186 31109 leader_election.cc:403] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:25:46.770543  3201 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:46.770157  3639 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.674032  3786 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.625393  3791 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:46.795635  3201 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:46.804730  3201 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.805308  3791 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:46.806383  3746 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.678s	user 0.004s	sys 0.008s
I20260709 12:25:46.807634  3786 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.773217  3760 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:46.808140 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2712) took 2308 ms. Trace:
I20260709 12:25:46.808471  3760 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.809209  3760 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:46.810704  3795 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 12:25:46.774673  3799 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 12:25:46.811537  3769 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:46.750941  3770 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:46.812109  3769 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:46.812588  3769 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:46.813839  3770 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:46.773242  3669 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:46.814620  3669 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.815366  3760 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.816636  3669 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.822451  3665 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.822980  3791 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:46.823429  3791 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:46.823801  3791 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.823964  3701 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:46.824529  3701 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.826440  3707 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:46.758200  3287 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.830674  3770 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.831249  3770 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.831801 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.793358 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1076) took 1273 ms. Trace:
I20260709 12:25:46.752075  3792 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.776758 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2588) took 2215 ms. Trace:
I20260709 12:25:46.835479 30986 rpcz_store.cc:276] 0709 12:25:44.560930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.561107 (+   177us) service_pool.cc:224] Handling call
0709 12:25:44.565714 (+  4607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.205101 (+639387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.776528 (+1571427us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b5400225954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:46.776699 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462572800}
I20260709 12:25:46.808565 31065 rpcz_store.cc:276] 0709 12:25:44.500144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.507772 (+  7628us) service_pool.cc:224] Handling call
0709 12:25:44.523974 (+ 16202us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.821392 (+297418us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:44.838201 (+ 16809us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.950439 (+112238us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.957122 (+  6683us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.957258 (+   136us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:44.957293 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.957358 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.444108 (+1486750us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:46.807837 (+363729us) raft_consensus.cc:1685] finished
0709 12:25:46.807942 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.808065 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":313468,"prepare.queue_time_us":17113,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":297813,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":5028,"spinlock_wait_cycles":9782656,"thread_start_us":111712,"threads_started":1,"wal-append.queue_time_us":134598}
W20260709 12:25:46.839555 31695 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
I20260709 12:25:46.840679 31695 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:46.851929  3770 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.852219  3788 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.853950  3575 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:46.854609  3792 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:46.857481  3721 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.754s	user 0.002s	sys 0.028s
W20260709 12:25:46.858572  3591 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.795s	user 0.008s	sys 0.011s
I20260709 12:25:46.724714  3250 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.816965  3760 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:25:46.860635  3250 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:46.833827 31696 rpcz_store.cc:276] 0709 12:25:45.519866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.520012 (+   146us) service_pool.cc:224] Handling call
0709 12:25:45.520455 (+   443us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:46.122458 (+602003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.122503 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:46.123215 (+   712us) log.cc:844] Serialized 52 byte log entry
0709 12:25:46.327239 (+204024us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:46.350304 (+ 23065us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.350570 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.350638 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.793084 (+442446us) raft_consensus.cc:1685] finished
0709 12:25:46.793111 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.793284 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":626057,"prepare.queue_time_us":22830,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":623256,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":8878,"spinlock_wait_cycles":74596608,"thread_start_us":203573,"threads_started":1,"wal-append.queue_time_us":526599}
I20260709 12:25:46.882701 31064 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:46.885314 31064 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:46.888118 31136 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:25:46.888522 31136 pending_rounds.cc:85] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 7
I20260709 12:25:46.888747 31136 pending_rounds.cc:107] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:25:46.888975 31136 raft_consensus.cc:2889] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:46.898258 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:46.857930  3721 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:46.908041 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 144665us
I20260709 12:25:46.905680 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.908901 31109 leader_election.cc:400] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:46.909170 31109 leader_election.cc:403] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:25:46.909415 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 415813us
I20260709 12:25:46.915469 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:46.926060 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:46.926146 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:46.928568 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:46.929280 30992 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:46.934276 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.936089 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:46.907877  3323 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:46.938535 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.939617 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:46.941972 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:46.943048 31696 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:46.915498  3701 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:46.948236 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.949875 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.952136 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:46.924432  3591 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
W20260709 12:25:46.954047 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45276.8us
I20260709 12:25:46.961134 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.967864 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:46.975291 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:46.976718 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:46.968544 31695 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:46.967382 31067 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:46.977926 31137 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:25:46.980852 30987 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 9.
I20260709 12:25:46.981709 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:46.982242 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.985072 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:46.985683 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:46.985733 31067 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:47.013564 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:46.915577  3647 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:46.971786 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 216037us
I20260709 12:25:46.942385  3769 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.021237 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:47.023679 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:46.963323  3817 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.963459  3358 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.052585 30987 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
W20260709 12:25:47.023304  3512 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.922s	user 0.021s	sys 0.003s
I20260709 12:25:47.076503 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:47.076699 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 55528.7us
I20260709 12:25:46.972157  3452 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.082325 30986 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:47.085151 31694 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:47.086285 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.019685  3486 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:46.944445  3323 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:46.963444  2860 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
W20260709 12:25:47.129009 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 46083.8us
I20260709 12:25:47.131211 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:47.131969 31060 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:47.135061 30992 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:25:47.135214 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.136489 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:47.147408  3792 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.149246  3323 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.096017  3452 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.091027  3591 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:47.090823  3639 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:47.085600  3817 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:47.166945 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 191471us
I20260709 12:25:47.155020  3591 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.159461  3808 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.160063  3452 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.121394  3809 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:47.185348 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:47.186759 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.187409 30992 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
W20260709 12:25:47.160084  3824 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:47.160181  3769 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:47.091550  3512 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.212437 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:47.122473  3638 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.091107  3358 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.180485  3591 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.184764  3711 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.236902 31063 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:47.140940  3486 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:47.235628 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 260849us
I20260709 12:25:47.242049 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:47.242879 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.161399  3791 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.244715 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:47.180136  3639 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.234632  3647 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.236586  3702 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.246351  3833 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:47.221468  3627 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:25:47.195854  3809 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.189141  3808 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.287562 31696 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45809554a739456ea0a596484ede5ac7"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:47.284493 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.302155 31696 raft_consensus.cc:606] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:47.302903 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:47.289038 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44396.5us
I20260709 12:25:47.225394  3769 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.325410  3769 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.290382  3702 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.326637  3735 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:47.326884  3512 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.327165  3735 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.327867  2860 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:47.328231  3702 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.330731 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.332088 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.333091 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.326171  3817 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.338264 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2428) took 1029 ms. Trace:
I20260709 12:25:47.338665 31140 rpcz_store.cc:276] 0709 12:25:46.308494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.623350 (+314856us) service_pool.cc:224] Handling call
0709 12:25:46.623746 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.663122 (+ 39376us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:46.930401 (+267279us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.133599 (+203198us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.141792 (+  8193us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.142254 (+   462us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:47.142344 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.142553 (+   209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.338049 (+195496us) raft_consensus.cc:1685] finished
0709 12:25:47.338072 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.338190 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55082,"prepare.queue_time_us":265223,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":4625,"raft.queue_time_us":109462,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":47505,"thread_start_us":306684,"threads_started":2,"wal-append.queue_time_us":382030}
I20260709 12:25:47.340956 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.244640  3721 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:47.298689  3832 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:47.343609 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.344213  3832 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:47.344333 31140 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:47.344717  3704 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.345206  3832 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.305603  3812 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:47.346411  3812 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.346803  3812 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:47.347008 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 744) took 2320 ms. Trace:
I20260709 12:25:47.347375 31693 rpcz_store.cc:276] 0709 12:25:45.026532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.070167 (+ 43635us) service_pool.cc:224] Handling call
0709 12:25:45.096493 (+ 26326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.915217 (+818724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.346732 (+1431515us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54006572d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:47.346926 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1881591552}
W20260709 12:25:47.306449  3819 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:25:47.351363 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2419) took 1466 ms. Trace:
I20260709 12:25:47.351776 30989 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:47.351684 31144 rpcz_store.cc:276] 0709 12:25:45.884851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.885665 (+   814us) service_pool.cc:224] Handling call
0709 12:25:45.896783 (+ 11118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.235568 (+338785us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18003a8c40. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb88cbfb 00007f2eeb89764b
0709 12:25:46.235716 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.235741 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.236190 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.675425 (+439235us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:46.675484 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.675641 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.351099 (+675458us) raft_consensus.cc:1685] finished
0709 12:25:47.351118 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.351306 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7924,"raft.queue_time_us":27300,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":12091,"spinlock_wait_cycles":25777408,"thread_start_us":469668,"threads_started":2,"wal-append.queue_time_us":776588}
I20260709 12:25:47.353609 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2237) took 1430 ms. Trace:
I20260709 12:25:47.306778  3753 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:47.354715  3753 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:47.355997 31696 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:47.355643  3753 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.356658 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2362) took 1481 ms. Trace:
I20260709 12:25:47.356768 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.357391 31140 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
W20260709 12:25:47.307168  3804 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:25:47.357054 30983 rpcz_store.cc:276] 0709 12:25:45.875410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.007681 (+132271us) service_pool.cc:224] Handling call
0709 12:25:46.008024 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.008424 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.008466 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.008973 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.353059 (+344086us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:46.353122 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.353207 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.356485 (+1003278us) raft_consensus.cc:1685] finished
0709 12:25:47.356503 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.356611 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":5206,"thread_start_us":325377,"threads_started":1,"wal-append.queue_time_us":1335909}
I20260709 12:25:47.359653 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:47.307937  3830 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:25:47.360613 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.361824 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2424) took 1449 ms. Trace:
I20260709 12:25:47.362205 31143 rpcz_store.cc:276] 0709 12:25:45.912451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.027335 (+114884us) service_pool.cc:224] Handling call
0709 12:25:46.027779 (+   444us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:46.292046 (+264267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.292079 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:46.292662 (+   583us) log.cc:844] Serialized 52 byte log entry
0709 12:25:46.625924 (+333262us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:46.725024 (+ 99100us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.757249 (+ 32225us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.126659 (+369410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.126774 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.337044 (+210270us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b18003debe0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:47.361632 (+ 24588us) raft_consensus.cc:1685] finished
0709 12:25:47.361653 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.361774 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5758,"prepare.queue_time_us":490670,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":96,"raft.queue_time_us":153219,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":228,"spinlock_wait_cycles":42379904,"thread_start_us":933860,"threads_started":4,"wal-append.queue_time_us":632545}
I20260709 12:25:47.353986 31136 rpcz_store.cc:276] 0709 12:25:45.923419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.060909 (+137490us) service_pool.cc:224] Handling call
0709 12:25:46.061257 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.889408 (+828151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.353407 (+463999us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54001e9bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:47.353554 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840613248}
I20260709 12:25:47.364574  3512 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.364943 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.365046  3512 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.365890  3788 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:47.308422  3639 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.366433  3788 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:47.308997  3816 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:25:47.310308  3358 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:47.311771  3638 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.367302  3512 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.313781  3809 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.315304  3486 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.368204  3486 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:47.323637  3791 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.238400  3711 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.368510  3788 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.368541  3486 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:47.369266  3486 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:47.370126  3711 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.372548  3734 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.373054  3734 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.374617  3672 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:47.375005  3672 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:47.353821 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:47.376588 31137 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:47.362102 31142 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:47.373191  3288 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:47.380414 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.381326 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:47.391034 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 369843us
W20260709 12:25:47.434291 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 56544.8us
I20260709 12:25:47.379935 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:47.469827 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 90332.6us
I20260709 12:25:47.475054 31693 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:47.479369 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2425) took 1566 ms. Trace:
I20260709 12:25:47.471606  3833 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:47.519944 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2231) took 2485 ms. Trace:
I20260709 12:25:47.483655  3734 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:47.532097  3591 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.469805  3672 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.563206 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.564121 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:47.564307 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.564677 31144 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.472703  3814 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:25:47.535606 31140 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:47.542166 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.547394 31135 rpcz_store.cc:276] 0709 12:25:45.034669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.869107 (+834438us) service_pool.cc:224] Handling call
0709 12:25:45.875258 (+  6151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.900662 (+ 25404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.900707 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.900762 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:45.900814 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.519693 (+1618879us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54003f48d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:47.519744 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.519882 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57874816}
I20260709 12:25:47.571089 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.571234 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.571830 31135 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:47.572252 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.572665 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.573277 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:47.573650 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.531258 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 937) took 2453 ms. Trace:
I20260709 12:25:47.574344 30984 rpcz_store.cc:276] 0709 12:25:45.077770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.077897 (+   127us) service_pool.cc:224] Handling call
0709 12:25:45.078150 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.487870 (+409720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.487905 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.487948 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:45.487989 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.530950 (+2042961us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b540014ebd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:47.530995 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.531180 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941853952}
W20260709 12:25:47.548947 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 168628us
I20260709 12:25:47.576341 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:47.576679 31140 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:47.576820  3591 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:47.577486  3591 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:47.577677 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:47.577862  3591 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:47.552063 31137 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:47.556687 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.538319 31139 rpcz_store.cc:276] 0709 12:25:45.912695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.030184 (+117489us) service_pool.cc:224] Handling call
0709 12:25:46.030624 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.098010 (+ 67386us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:46.098041 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.098505 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.294086 (+195581us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:46.294125 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.478987 (+1184862us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b54003f4dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:47.479034 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.479078 (+    44us) raft_consensus.cc:1685] finished
0709 12:25:47.479090 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.479296 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":304,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":154062080,"thread_start_us":195169,"threads_started":1,"wal-append.queue_time_us":254828}
I20260709 12:25:47.586019 31144 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
W20260709 12:25:47.581029 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 201144us
I20260709 12:25:47.577502 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.563314  3672 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:47.471735  3815 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:47.533860 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.566386 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:47.600149 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.561651  3788 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.561316  3639 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:25:47.618325  3401 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:47.618573 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.618674 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.618888  3401 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.619297 31695 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:47.618062 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.565337  3712 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:25:47.620959  3401 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:47.621455  3836 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.486s	user 0.002s	sys 0.002s
I20260709 12:25:47.621635  3712 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:47.622685 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:47.623010  3805 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.853s	user 0.000s	sys 0.002s
I20260709 12:25:47.624212 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 750) took 1724 ms. Trace:
I20260709 12:25:47.624527 31691 rpcz_store.cc:276] 0709 12:25:45.900202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.900348 (+   146us) service_pool.cc:224] Handling call
0709 12:25:45.900743 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.179475 (+278732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.179502 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.191865 (+ 12363us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.445860 (+253995us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:46.447570 (+  1710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.447843 (+   273us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.622488 (+1174645us) spinlock_profiling.cc:243] Waited 7.86 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:47.623954 (+  1466us) raft_consensus.cc:1685] finished
0709 12:25:47.623972 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.624160 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":296419,"raft.queue_time_us":295742,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":64,"spinlock_wait_cycles":18086656,"thread_start_us":242420,"threads_started":1,"wal-append.queue_time_us":517934}
I20260709 12:25:47.569825 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2365) took 1692 ms. Trace:
I20260709 12:25:47.627140  3760 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.627740  3760 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.627082 30988 rpcz_store.cc:276] 0709 12:25:45.877130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.125749 (+248619us) service_pool.cc:224] Handling call
0709 12:25:46.126009 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.126188 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:46.182369 (+ 56181us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.569158 (+1386789us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b18004eece0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf
0709 12:25:47.569292 (+   134us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.569475 (+   183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.569524 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:47.569574 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.569659 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.569771 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":108441,"prepare.queue_time_us":627953,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":191927,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9540,"spinlock_wait_cycles":79740288,"thread_start_us":327766,"threads_started":2,"wal-append.queue_time_us":910221}
I20260709 12:25:47.628846  3808 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:47.631026 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:47.587595 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:47.634899 30984 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:47.635270 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:47.635845 31063 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:47.636721 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:47.637316 31060 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.581707  3833 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:47.629860 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2720) took 2789 ms. Trace:
I20260709 12:25:47.638619  3833 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.639051  3833 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:47.638808 31058 rpcz_store.cc:276] 0709 12:25:44.840027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.972885 (+132858us) service_pool.cc:224] Handling call
0709 12:25:44.973359 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.097003 (+123644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.629654 (+2532651us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54001a72d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:47.629808 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249747712}
I20260709 12:25:47.640508 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:47.597105 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 55009us
I20260709 12:25:47.642890 31661 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.642908 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.611159 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.611985 31140 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:47.644338 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.560077 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.614152 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:47.594362 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58591.1us
I20260709 12:25:47.645320 31144 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:47.645767 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.646418 30983 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
W20260709 12:25:47.647027 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 87057us
I20260709 12:25:47.610212 31135 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:47.647539 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.647730 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.648487 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:47.649659 30988 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:47.650951 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:47.651696 30988 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.643590 30985 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:47.655175 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:47.657038 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.651083 30984 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:25:47.658528 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:47.660243 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:47.661365 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.654019 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.656136 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.662205 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:47.664255  3640 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:47.664665  3640 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.664994 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.665172  3640 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.665827 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.666397  3802 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:47.666906 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:47.661736  3817 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:47.667958 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:47.661567  3773 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:47.661845  3290 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:47.669555 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.654533  3815 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.669840  3358 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:47.670001 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 891) took 2853 ms. Trace:
I20260709 12:25:47.664587 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.670596  3358 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:47.666936  3802 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:47.671535  3815 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:47.654085  3808 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.654778  3788 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.672137  3802 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.672536  3808 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.672928  3160 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:47.674079  3788 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.674355 31140 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:47.674618 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.675710  3679 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:47.677067  3755 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:47.678934  3833 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.680331 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.681959 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:47.683105 31063 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:47.683863  3833 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.622049  3712 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:47.685614  3712 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.687279 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.687482  3290 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:47.596927  3814 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:47.688364  3358 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:47.688503  3814 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:47.688971  3358 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:47.689924  3358 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.582032  3826 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:47.691139  3826 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.691480  3826 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:47.693872 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:47.597327  3634 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:47.694499 31061 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:47.689640  3814 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.695676  3814 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.629909  3760 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.698618 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.699790 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.700223 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.701850 31137 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:47.643617  3691 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:25:47.712381  3703 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:47.713006  3691 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:47.713384  3691 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:47.714190  3691 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.715474  3691 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.682410 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:47.718760 31064 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:47.719861 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:47.668751 31060 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
W20260709 12:25:47.702824  3807 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.582s	user 0.002s	sys 0.000s
I20260709 12:25:47.703871 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:47.614872  3591 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:47.723876  3755 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:47.724342  3755 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.709353  3581 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:47.665273 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.710255  3201 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:47.711601 30983 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.711691 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:47.712026 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.727327 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:47.727694 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.709013 31691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:47.750563 31691 raft_consensus.cc:606] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.750859 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.727907 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:47.728502 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68357.3us
I20260709 12:25:47.752969 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:47.728816 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 72778.9us
W20260709 12:25:47.735468 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.736799 31061 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:25:47.681900  3714 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.738278  3838 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:47.670485 31141 rpcz_store.cc:276] 0709 12:25:44.816073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.816203 (+   130us) service_pool.cc:224] Handling call
0709 12:25:44.816497 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.405989 (+589492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.669738 (+2263749us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b540001ac54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:47.669917 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1309279104}
I20260709 12:25:47.763410 31140 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:25:47.738715  3808 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:47.615605  3486 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.739234  3712 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.663872 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.677928  3735 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:47.771363 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 107588us
I20260709 12:25:47.772516 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.740980 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:47.742744 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:47.744982  3672 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:47.757501 30989 raft_consensus.cc:2393] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:47.759171 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:47.776536 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.747746 30992 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:47.766829  3643 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:47.767140 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:47.781684 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.774747 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.760891 30986 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 13: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:47.787171 30985 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:47.748914  3817 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.781687 30988 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:47.585711  3812 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.744683  3755 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:47.788308 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.796025 30983 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:47.798424 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:47.799008  3770 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:25:47.793113  3588 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.767s	user 0.018s	sys 0.008s
I20260709 12:25:47.800120  3638 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:47.805680 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:47.807027 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:47.810319 31697 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:25:47.810804 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 82976.9us
I20260709 12:25:47.820101 31691 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:47.810676 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:47.798499 30988 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:25:47.821821 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:47.822486 30988 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:47.815789 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.776103  3714 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:47.823494 30992 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:47.779249  3711 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:47.739579  3826 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.799012  3832 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:47.829082 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.829447 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:47.756744  3581 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:47.789216  3833 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:47.803606  3721 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:47.796909  3742 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:47.798498  3634 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.773800  3255 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:25:47.852283 30988 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:25:47.809937  3533 raft_consensus.cc:2764] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
W20260709 12:25:47.825111  3588 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.803s	user 0.019s	sys 0.008s
I20260709 12:25:47.853590 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:47.862816 30989 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:47.867486 31140 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:47.872143 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:47.874867 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 69284.1us
I20260709 12:25:47.876199 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:47.877490 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:47.878170 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 46933.6us
I20260709 12:25:47.879694 30990 raft_consensus.cc:2368] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12.
I20260709 12:25:47.883939 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:47.843756  3323 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:47.860376 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:47.803417  3817 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.896831 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.896590 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:47.904610 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44285us
I20260709 12:25:47.905187 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.911761  3853 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:47.913589  3833 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:47.914100  3833 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.914543  3833 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:47.920033  3855 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:47.922210 31137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51358
I20260709 12:25:47.922942 31137 raft_consensus.cc:606] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:47.924144  3735 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:47.925043 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.926517 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:47.927623 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:47.927727 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:47.880280  3639 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:47.883004  3679 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:47.894277  3781 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:47.942464  3679 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:47.942780  3781 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:47.943058  3679 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:47.943861  3781 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.889626  3791 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:47.946521  3791 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.946939  3791 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.889436  3753 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:25:47.894901  3721 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:47.952487  3721 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.939262  3846 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:47.959508  3639 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:47.960019 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 76184.3us
I20260709 12:25:47.962112  3647 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.963244  3812 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.938366  3250 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:25:47.967839  3640 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:47.968838 31138 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:47.938944  3822 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:47.934830  3854 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:25:47.974367 30984 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:47.977392 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:47.978114 31064 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.984635 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:47.893890  3323 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:47.994582  3323 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:48.006269 31060 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:47.994814  3591 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.010591 31691 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:25:47.970002  3640 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:48.022267 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:48.024657 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:48.046087 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 149315us
I20260709 12:25:48.026034 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:48.033179 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 121239us
I20260709 12:25:48.047372 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:47.902621  3643 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:47.936645  3581 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.962114  3639 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:47.939857  3486 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:47.938043  3826 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:48.099036 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 76891.2us
W20260709 12:25:48.100462 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 53147.4us
I20260709 12:25:48.090979  3717 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:48.107096  3717 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.109298  3717 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:48.027204  3770 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:48.121575  3791 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.113055 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:48.116122 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.114075  3817 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:48.110611  3840 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:48.122321  3379 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:48.126682 31065 raft_consensus.cc:2410] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:25:48.070605  3283 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:48.129220 31063 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:48.114318  3643 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.139948  3840 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.140861  3703 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.141426  3703 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.143558  3703 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:48.144984  3710 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.145468  3710 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.146726  3634 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.121599  3647 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:48.140101  3643 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:48.148974  3861 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:25:48.150063  3643 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:48.137998 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:48.135654 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:48.159273 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:48.115778  3769 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:48.131008  3581 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:48.070091  3754 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:25:48.165138 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:48.131242  3717 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:48.168618 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:48.169246 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:48.171238 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 70889us
I20260709 12:25:48.174945 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:48.159545  3714 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:48.157584  3817 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:48.157454  3643 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:48.160511  3826 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:48.161969  3846 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:48.164990  3838 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:48.216732  3857 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.349s	user 0.002s	sys 0.000s
I20260709 12:25:48.222914  3591 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:48.175974  3717 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.225117  3717 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:48.226341  3679 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.182783  3255 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:48.230593  3255 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:48.232049 31060 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:25:48.151057  3770 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.233877  3770 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:48.162079  3250 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:25:48.212260 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1284) took 3379 ms. Trace:
I20260709 12:25:48.182955  3643 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.235142  3791 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:48.235369  3643 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:48.155813  3755 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.231603  3255 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.241117  3255 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:48.200337  3826 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.234467 31059 rpcz_store.cc:276] 0709 12:25:44.833051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.960321 (+127270us) service_pool.cc:224] Handling call
0709 12:25:47.630295 (+2669974us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.630636 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.630664 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.631161 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.897056 (+265895us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:47.897176 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.897258 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.211080 (+313822us) raft_consensus.cc:1685] finished
0709 12:25:48.211100 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.212026 (+   926us) spinlock_profiling.cc:243] Waited 2.65 s on lock 0x7b54001a72d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:48.212200 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7812,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":6105024384,"thread_start_us":264323,"threads_started":1,"wal-append.queue_time_us":529372}
I20260709 12:25:48.243044  3826 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:48.243592 31696 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 13. Ops: []
I20260709 12:25:48.203572  3782 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:48.246711  3782 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:48.176829  3802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.003s	sys 0.006s
W20260709 12:25:48.258517 31062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.005s	sys 0.029s
I20260709 12:25:48.174902  3581 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:48.260089 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.183557  3817 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:48.261852 31696 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:48.266744 31061 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
W20260709 12:25:48.259470 31062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.005s	sys 0.029s
I20260709 12:25:48.270371 31696 raft_consensus.cc:606] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:48.249289  3782 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:48.267972  3665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.018s	sys 0.007s
I20260709 12:25:48.270936 31696 raft_consensus.cc:614] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:48.256263  3802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.003s	sys 0.006s
W20260709 12:25:48.272500 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 311625us
I20260709 12:25:48.289446 31064 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:48.266857  3817 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:48.300693 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:48.301468 31064 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.301936 31062 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:48.308542  3734 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:48.309213 31060 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:48.310828 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:48.312013  3814 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:48.312355 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.312505  3814 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:48.313275 31060 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:48.313781  3814 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.316409  3638 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:48.321007  3287 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:48.300858 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:48.276504  3665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.019s	sys 0.008s
I20260709 12:25:48.332890 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:48.299765  3358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.004s	sys 0.008s
I20260709 12:25:48.330358 31064 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:48.275940  3679 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:48.339051 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.340859 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.339246  3734 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.347681 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 46907.6us
W20260709 12:25:48.338299  3358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.005s	sys 0.008s
I20260709 12:25:48.349094 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.351631 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.345762 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:25:48.345160  3740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.380s	user 0.017s	sys 0.030s
I20260709 12:25:48.332768  3588 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:25:48.339040  3791 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:25:48.422847 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.361042 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:48.434373 31135 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:48.369329 31140 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:48.388995 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:48.436609 31139 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:48.355930 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:48.446385 31140 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:48.389086 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:48.448202 31135 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:48.387619  3323 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:48.366412  3860 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:25:48.424077  3788 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:25:48.441494  3740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.658s	user 0.024s	sys 0.032s
I20260709 12:25:48.456290 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:48.358009  3781 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:48.370766 31693 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:48.451566 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:48.473551 31140 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:25:48.458828 31058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.015s	sys 0.013s
W20260709 12:25:48.364063  3858 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:25:48.459900 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:48.368530  3815 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.474159 31058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.015s	sys 0.013s
I20260709 12:25:48.461902 31135 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:48.464293 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.464491 31068 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:48.460074 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 383996us
I20260709 12:25:48.468202 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:48.468596 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:48.460824 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:48.482386 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:48.517885 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.562922 31140 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:48.348104  3734 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:48.572896 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 86175us
I20260709 12:25:48.574131 31690 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:48.577994 31139 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:48.365052  3643 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.580623 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:48.581399 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 99089us
I20260709 12:25:48.582156 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:48.444089  3791 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:48.575472  3833 consensus_meta.cc:287] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.660s	user 0.008s	sys 0.003s
I20260709 12:25:48.584860 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:48.586325 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:48.461670  3788 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:48.567970  3490 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:48.349272  3581 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.595039 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:48.595368 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:48.603960 31135 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:25:48.605345  3379 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:48.605790 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 1288) took 1725 ms. Trace:
I20260709 12:25:48.605885  3379 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:48.606208 31065 rpcz_store.cc:276] 0709 12:25:46.880434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.886190 (+  5756us) service_pool.cc:224] Handling call
0709 12:25:48.605585 (+1719395us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002a3d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:48.605734 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821743360}
I20260709 12:25:48.606807  3379 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.612249 31067 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:25:48.612653 31067 pending_rounds.cc:85] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 11
I20260709 12:25:48.612869 31067 pending_rounds.cc:107] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:25:48.613093 31067 raft_consensus.cc:2889] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:48.614967  3872 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 12:25:48.622809  3874 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:25:48.631310  3770 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.631837  3770 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.635170 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 916) took 1346 ms. Trace:
I20260709 12:25:48.635562 31143 rpcz_store.cc:276] 0709 12:25:47.289077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.364049 (+ 74972us) service_pool.cc:224] Handling call
0709 12:25:47.364344 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.374890 (+ 10546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.374923 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.375351 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.739969 (+364618us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:47.740010 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.740082 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.634961 (+894879us) raft_consensus.cc:1685] finished
0709 12:25:48.634984 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.635112 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10008,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":364243,"threads_started":1,"wal-append.queue_time_us":797265}
I20260709 12:25:48.643358 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:48.633949  3770 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:48.661599 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 66294.6us
I20260709 12:25:48.663115 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:48.687352 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2242) took 1336 ms. Trace:
I20260709 12:25:48.687884 31136 rpcz_store.cc:276] 0709 12:25:47.351304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.366389 (+ 15085us) service_pool.cc:224] Handling call
0709 12:25:47.366663 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.376588 (+  9925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.376619 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.470548 (+ 93929us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.751568 (+281020us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:47.751612 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.751675 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.666290 (+914615us) raft_consensus.cc:1685] finished
0709 12:25:48.666309 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.687289 (+ 20980us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9551,"raft.queue_time_us":9964,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":37144,"thread_start_us":239659,"threads_started":1,"wal-append.queue_time_us":717595}
I20260709 12:25:48.592386  3588 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:48.569605  3815 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.599007  3201 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:48.591789  3879 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:48.691608 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.720655  3714 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:48.656358  3283 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:25:48.704856 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 62403.2us
I20260709 12:25:48.756306 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:48.601992  3665 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.579495  3647 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:25:48.712481 31696 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:48.597844  3358 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260709 12:25:48.715131 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:48.688365  3486 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:48.753360 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:48.760118 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61909.8us
I20260709 12:25:48.763618 31142 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:25:48.714051 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:48.603157  3833 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.768867 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:48.720129  3634 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:48.687938  3287 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:25:48.686645  3853 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:25:48.687513  3802 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:25:48.724509  3879 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.748782  3714 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:48.753770  3201 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:48.766709  3283 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.715898  3588 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.768615  3255 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:25:48.795130  3826 consensus_meta.cc:287] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:25:48.770560  3486 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:48.818552 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1306) took 1099 ms. Trace:
I20260709 12:25:48.850811 31058 rpcz_store.cc:276] 0709 12:25:47.719329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.720613 (+  1284us) service_pool.cc:224] Handling call
0709 12:25:47.720890 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.721208 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.721235 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.721714 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.481249 (+759535us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:48.481287 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.481349 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.818321 (+336972us) raft_consensus.cc:1685] finished
0709 12:25:48.818344 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.818492 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1080,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":91,"thread_start_us":759135,"threads_started":1,"wal-append.queue_time_us":1092739}
I20260709 12:25:48.820731  3710 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:48.821602  3881 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:25:48.716503  3703 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:25:48.857460 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:48.836736  3714 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.858603 31697 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:48.828855  3879 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:48.837257  3769 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:48.864135  3815 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:48.862244  3486 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.874675  3791 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:48.877594  3557 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:48.897204  3691 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:48.902029  3557 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:48.874179  3323 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:48.862527  3826 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.837929  3634 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:48.903980  3691 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:48.931330 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2603) took 1955 ms. Trace:
I20260709 12:25:48.931838 30987 rpcz_store.cc:276] 0709 12:25:46.975847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.089845 (+113998us) service_pool.cc:224] Handling call
0709 12:25:47.090242 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.375829 (+285587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.375870 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.469942 (+ 94072us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.819210 (+349268us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:47.867494 (+ 48284us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.012949 (+145455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.930871 (+917922us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54000230d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:48.930934 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.930993 (+    59us) raft_consensus.cc:1685] finished
0709 12:25:48.931102 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.931259 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":339695,"prepare.queue_time_us":48335,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":201916,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":88551,"spinlock_wait_cycles":530410752,"thread_start_us":285777,"threads_started":1,"wal-append.queue_time_us":496682}
I20260709 12:25:48.935050 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2611) took 1287 ms. Trace:
I20260709 12:25:48.935541 30990 rpcz_store.cc:276] 0709 12:25:47.647934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.648291 (+   357us) service_pool.cc:224] Handling call
0709 12:25:48.934825 (+1286534us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54001f7054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:48.934984 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530171648}
W20260709 12:25:48.951407  3870 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.562s	user 0.003s	sys 0.001s
I20260709 12:25:48.963968 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1303) took 1284 ms. Trace:
I20260709 12:25:48.964478 31067 rpcz_store.cc:276] 0709 12:25:47.679507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.680474 (+   967us) service_pool.cc:224] Handling call
0709 12:25:48.611915 (+931441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.613517 (+  1602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.963714 (+350197us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54002a3d50. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:48.963893 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141557632}
I20260709 12:25:48.912309  3643 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:48.971894 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.976122 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:48.921751  3734 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:48.974210  3638 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.059s	sys 0.081s
W20260709 12:25:48.991896  3638 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.059s	sys 0.082s
I20260709 12:25:48.776144  3788 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:48.985252 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:48.993404  3634 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:48.978106  3512 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.979318  3832 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.911873  3201 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:49.033545  3883 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:49.035667 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 921) took 1292 ms. Trace:
I20260709 12:25:49.044884  3840 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.036135 31144 rpcz_store.cc:276] 0709 12:25:47.743523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.743629 (+   106us) service_pool.cc:224] Handling call
0709 12:25:47.768016 (+ 24387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.770934 (+  2918us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:47.788981 (+ 18047us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.180189 (+391208us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b18004cc360. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:25:48.569786 (+389597us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.578117 (+  8331us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.578286 (+   169us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:48.578379 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.578443 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.035317 (+456874us) raft_consensus.cc:1685] finished
0709 12:25:49.035436 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.035591 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10677,"prepare.queue_time_us":17945,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":34,"raft.queue_time_us":49543,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":4949,"spinlock_wait_cycles":43392896,"thread_start_us":381239,"threads_started":1,"wal-append.queue_time_us":1099120}
I20260709 12:25:49.052567  3588 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:49.054859  3283 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:49.056099  3882 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.285s	user 0.002s	sys 0.000s
I20260709 12:25:49.026067  3832 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.062996 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 958) took 1323 ms. Trace:
I20260709 12:25:49.066269 30991 rpcz_store.cc:276] 0709 12:25:47.739727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.791157 (+ 51430us) service_pool.cc:224] Handling call
0709 12:25:47.791424 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.791742 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.791769 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.802199 (+ 10430us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.161934 (+359735us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:48.161973 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.162057 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.950720 (+788663us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:49.030498 (+ 79778us) raft_consensus.cc:1685] finished
0709 12:25:49.030526 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.062903 (+ 32377us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39210,"raft.queue_time_us":15848,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":35411,"spinlock_wait_cycles":5169024,"thread_start_us":294711,"threads_started":1,"wal-append.queue_time_us":505769}
I20260709 12:25:49.027508  3833 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:49.058185  3817 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.726s	user 0.005s	sys 0.003s
I20260709 12:25:49.024245  3512 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.062444  3832 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:49.091630 31107 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.035s	sys 0.054s
I20260709 12:25:49.020112  3691 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.018723  3781 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:49.097743 31107 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.035s	sys 0.054s
I20260709 12:25:49.025754  3557 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:49.113445 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 337796us
I20260709 12:25:49.117928 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:49.121623 31140 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.026s	sys 0.036s
W20260709 12:25:49.123026 31140 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.026s	sys 0.036s
I20260709 12:25:49.127657  3283 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:49.131902 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.140902  3853 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:49.142416 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.145164 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.091363  3817 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.149925 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.149987 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.109504  3832 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:49.155198 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.155635 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.120496  3691 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:49.120252  3826 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.126152  3672 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:49.148653  3833 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.171475 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:49.159184  3283 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.147557  3557 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.152223  3880 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.153436  3853 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:49.148465  3808 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:49.177696 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:49.177798 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 421581us
I20260709 12:25:49.161597  3802 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:49.178087 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.170715 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.178350 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.192155 31144 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:49.179612 31136 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:49.166079  3840 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.186443 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.192787 30991 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:49.182897  3773 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.195770 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.196493 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.196607 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.197595 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.183228 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.198690 31138 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:49.192689 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.193557 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.189515 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:49.175573  3672 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:49.204046 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.204489  3672 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:49.204802  3817 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:49.205384  3899 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:49.205911  3899 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.206338  3710 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:25:49.192240  3886 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:25:49.176121  3769 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:25:49.209362  3769 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:49.209790  3486 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.213825 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.165371  3832 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.212158  3769 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.218962  3490 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:49.219156 31141 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:49.222707 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.223351  3899 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.226953 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:49.229791 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.247931 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.199186  3283 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:25:49.246168  3769 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.246467  3490 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.247308  3782 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:49.247833  3817 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:49.247231  3591 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:49.260118  3893 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:49.260583 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.262763 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.263727 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.250236 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:49.341235 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 91088.8us
I20260709 12:25:49.256548 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.273960 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:49.345759 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 71884.7us
I20260709 12:25:49.286900  3714 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.184007  3781 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.177448  3833 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.290369 31060 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:49.215135  3255 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:49.250496  3672 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:49.348810 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.292622  3832 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.195045  3808 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.295778 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.327657 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:49.330086 31063 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:49.359197 31063 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:49.251381  3904 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:49.335804 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.256752  3533 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.323659  3283 consensus_queue.cc:1243] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:49.359434 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.372292 30988 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:49.374866 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:49.382532 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48592.4us
W20260709 12:25:49.383786 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.339924  3591 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:49.389038 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:49.393146 31141 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:49.355101  3759 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:49.336920  3846 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:49.348440  3815 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:49.348426  3581 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:49.396111 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.397037 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.406391 31695 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:49.369194  3880 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.397866 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.370766  3832 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:49.363699  3900 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.409458 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.336581  3899 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:25:49.415230 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.424706 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:49.417260 31144 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:49.384874  3201 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.407277 31689 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:49.347623  3817 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.426486 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.427593 31144 raft_consensus.cc:2410] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:49.428431  3791 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.429328  3791 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:49.429816  3791 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.430269  3791 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:49.355494  3588 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:25:49.340385  3782 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.400409  3533 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:25:49.406737  3846 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.389395  3826 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:25:49.354703  3893 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.459286 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.459811 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:25:49.471637  3876 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.047s	user 0.001s	sys 0.002s
I20260709 12:25:49.455227  3250 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:49.494899 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.443612  3833 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.449920  3898 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:49.475168  3906 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.002s
I20260709 12:25:49.451416  3817 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:49.444260  3840 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:49.502781 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 123429us
W20260709 12:25:49.485740  3910 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
I20260709 12:25:49.506994 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.422369  3581 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.439085  3918 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:25:49.483703  3255 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:49.444993  3900 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.500109  3899 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:49.435976  3832 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.522861 30990 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:49.441109  3591 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.499205  3672 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:49.537618  3901 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:49.554397 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 47499.5us
I20260709 12:25:49.553030 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1095) took 1731 ms. Trace:
I20260709 12:25:49.516647  3899 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:49.552026  3901 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:49.555240 31698 rpcz_store.cc:276] 0709 12:25:47.821764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.822696 (+   932us) service_pool.cc:224] Handling call
0709 12:25:47.822957 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.972340 (+149383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.972370 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.972819 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.268561 (+295742us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:48.268603 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.268664 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.441325 (+1172661us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:49.534340 (+ 93015us) raft_consensus.cc:1685] finished
0709 12:25:49.534370 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.552944 (+ 18574us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149026,"raft.queue_time_us":149537,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":7805696,"thread_start_us":295385,"threads_started":1,"wal-append.queue_time_us":443871}
I20260709 12:25:49.556708  3901 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.558005 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:49.558183  3901 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:49.558794 30989 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:49.560825  3832 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.561190  3581 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.562392  3773 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:49.562922 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.564311 31137 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:49.566609 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:49.567150 30987 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:49.566157 31137 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.556636  3899 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.573084  3899 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:49.573378  3921 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:49.576521 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:49.577693 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:49.578205 30989 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:49.573889  3921 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.585806 30987 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:49.586697 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:49.590250 30989 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:49.591171 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.510300  3840 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.550633  3672 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:49.509980  3250 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.516170  3255 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.510130  3898 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.607532 30989 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:49.598101  3755 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:49.606963 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.608614  3672 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:49.624749 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.617048  3808 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.627239 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.615840  3714 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.642861  3880 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.643330  3875 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:49.643543  3879 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.769s	user 0.012s	sys 0.000s
I20260709 12:25:49.628010  3802 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:49.630494  3255 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.632094  3250 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.636824 30987 raft_consensus.cc:2410] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:25:49.645447  3898 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.643774  3875 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:49.620119  3769 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.638760  3919 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.647001  3840 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.640880 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2259) took 1190 ms. Trace:
I20260709 12:25:49.647492 31140 rpcz_store.cc:276] 0709 12:25:48.462344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.564117 (+101773us) service_pool.cc:224] Handling call
0709 12:25:48.564467 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.124214 (+559747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.124260 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.160951 (+ 36691us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.202404 (+ 41453us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:49.202454 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.202549 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.640681 (+438132us) raft_consensus.cc:1685] finished
0709 12:25:49.640702 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.640826 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5682,"raft.queue_time_us":43674,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":68,"thread_start_us":584291,"threads_started":2,"wal-append.queue_time_us":371713}
I20260709 12:25:49.649063 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.646615  3875 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.650712  3893 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.650969  3875 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:49.651283  3782 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:49.652642  3512 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.651294  3846 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.654527  3791 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.655390 31063 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.656539 31063 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:49.656689  3791 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.646768  3919 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.662691 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:49.663570  3921 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.639310  3900 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.663609 30989 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:49.650110 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.667171 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:49.668538 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:49.669795 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:49.671325 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 44064.5us
I20260709 12:25:49.628929  3773 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:49.689090 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.689322 30989 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:49.690140 31063 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:49.690502 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1093) took 2046 ms. Trace:
I20260709 12:25:49.693382 30991 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:49.693439 31693 rpcz_store.cc:276] 0709 12:25:47.643886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.668849 (+ 24963us) service_pool.cc:224] Handling call
0709 12:25:47.726360 (+ 57511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.472390 (+746030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.922361 (+449971us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400512a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:49.690252 (+767891us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b5400512a50. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:49.690425 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612935296}
I20260709 12:25:49.694942 30989 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:49.678925  3878 raft_consensus.cc:993] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:49.703163 30989 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:49.643980  3879 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:49.708664 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.710659 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.710703 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 923) took 1351 ms. Trace:
I20260709 12:25:49.711113 31142 rpcz_store.cc:276] 0709 12:25:48.359689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.359811 (+   122us) service_pool.cc:224] Handling call
0709 12:25:48.389521 (+ 29710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.774879 (+385358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.710497 (+935618us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540012cfd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:49.710650 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859408256}
I20260709 12:25:49.713943  3813 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:49.714098  3879 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.714427  3813 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:49.715305  3813 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.716296  3813 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.725040  3672 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.736675 31136 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:49.761703 31143 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 12:25:49.782369  3788 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:49.786126  3702 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:49.787652 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2448) took 1216 ms. Trace:
W20260709 12:25:49.788455 30955 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.66:45157 timed out after 1.631s (SENT) [suppressed 9 similar messages]
I20260709 12:25:49.788163 31068 rpcz_store.cc:276] 0709 12:25:48.571234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.571370 (+   136us) service_pool.cc:224] Handling call
0709 12:25:48.571650 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.583049 (+ 11399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.583090 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.600992 (+ 17902us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.764441 (+1163449us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b18001c5c40. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:49.785902 (+ 21461us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:49.785944 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.786034 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.787346 (+  1312us) raft_consensus.cc:1685] finished
0709 12:25:49.787364 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.787597 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11006,"raft.queue_time_us":11131,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":11042,"spinlock_wait_cycles":43996928,"thread_start_us":21273,"threads_started":1,"wal-append.queue_time_us":1167329}
I20260709 12:25:49.810765  3672 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.785002  3919 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.765398  3490 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.819517  3490 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.825479  3634 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.825968  3634 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.829162  3634 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.833426  3379 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:49.834376 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:49.835884 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.839840  3742 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.015s	sys 0.005s
W20260709 12:25:49.840238  3742 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.015s	sys 0.005s
I20260709 12:25:49.840708  3742 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.843736 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.845556  3379 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:49.845867 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.846256  3379 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:49.846598 31696 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:25:49.847615 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:49.849893 31068 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:49.850263 31068 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:49.847354  3379 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.851854 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:49.854866 31068 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.856606 31068 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.855665  3638 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 12:25:49.858922 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.843007  3742 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:49.854243  3679 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:49.862363  3679 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.862720  3679 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:49.863776  3712 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:25:49.871364  3742 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:49.871735 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.880976 31065 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:49.886307 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 391482us
I20260709 12:25:49.888055 31109 leader_election.cc:400] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12.
I20260709 12:25:49.888392 31109 leader_election.cc:403] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:25:49.891002 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.896733 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.902436  3634 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:49.908627 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.907006 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:49.914022 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.921559 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.937117 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.937736 31065 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:25:49.941210 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:49.942157 31060 raft_consensus.cc:2410] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:25:49.942766  3901 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.943208 31060 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:49.894251  3833 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.944205 31067 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:49.944232 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.948746 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.948881 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:49.953001 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.952641 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.952725 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:49.957252 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:49.958429 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.960762 31065 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 12:25:49.911994  3826 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:49.936252  3817 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12.
I20260709 12:25:49.938818  3512 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:49.943776  3490 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:49.952320  3919 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.956718  3932 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.959439  3832 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:49.964958  3691 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:49.943238  3901 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.988628 31058 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:49.989262 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 99902.8us
I20260709 12:25:49.987984 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.990010 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:49.990669 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:49.980476  3379 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.047674 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 59136.4us
I20260709 12:25:50.012564 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.018113 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:49.966267  3929 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:25:50.048574 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57988.5us
W20260709 12:25:50.039603 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 51664.8us
I20260709 12:25:50.049851 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:50.019984 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.051823 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.052007 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.052170 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.017308 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.053885 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.017747 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.008663  3815 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:50.016112  3893 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:50.020221  3581 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:50.022404  3323 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:25:50.019129  3283 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.994478  3672 raft_consensus.cc:1081] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:50.008920  3853 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 12:25:50.046723  3948 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:50.050390 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.054025 31140 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 12:25:49.983772  3711 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:50.016690 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.057194 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.099830 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.100950 31695 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:25:50.101317 31695 pending_rounds.cc:85] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 12
I20260709 12:25:50.101549 31695 pending_rounds.cc:107] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:25:50.101778 31695 raft_consensus.cc:2889] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:50.102142  3894 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:50.102830  3894 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:50.103485  3894 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:25:50.034698  3926 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:25:50.009009  3943 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:25:50.059412  3918 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:49.988969  3813 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:50.107434 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.108089 31141 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:50.108598  3672 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:50.097630  3711 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:50.110996  3711 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.110677  3679 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.112705  3679 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:49.982985  3512 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.113641 31063 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:50.114558 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.115661 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.116196 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:50.064929  3638 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:50.118261  3638 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:50.119148  3638 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.119903  3512 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.119764  3711 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.066411  3486 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.060500  3913 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:50.072067 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.074283 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.058323  3640 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:50.056195  3691 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:50.075450 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.986593  3919 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.056151  3379 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:49.991433  3832 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.049990  3929 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.049618  3880 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:50.056344  3808 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:50.087800 30984 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:50.201093 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:50.201864  3703 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:50.202414 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.203567 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:50.204137  3703 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:50.204602 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.088603 31698 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:50.089354 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:50.097851  3853 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:50.037132  3901 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:50.113638  3875 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:50.120355  3638 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.984014  3490 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.100972 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:50.121987 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.125110 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:50.170775  3581 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:50.126390 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:50.251060 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 79501.8us
I20260709 12:25:50.127315 30989 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:50.097604 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.128187 30991 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:25:50.129741 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.252748 30986 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:50.130112  3782 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:50.130453 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:50.131515 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.132133 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:50.132843 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:50.258471 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.259727 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.259974  3921 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:50.260568  3921 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:50.170632 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:50.261497  3921 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.180245 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.181793 31058 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:50.262576  3921 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.187461 31144 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:50.188956 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:50.263028 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 82866.8us
I20260709 12:25:50.263262 31692 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:50.264966  3913 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:50.265337  3913 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:50.265697  3913 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.190286  3919 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.185854  3358 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.272130  3358 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.273108 31689 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:50.193938  3379 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:50.195544 31140 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:50.274724  3358 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.195633  3832 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:50.285197 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:50.186204  3928 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:50.196731  3929 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:50.206133 30984 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:50.240974 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.242748 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.289991 31063 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
W20260709 12:25:50.243788 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.125748 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.250684 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:50.124379  3893 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.178371  3899 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:50.063982  3283 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.204615  3703 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:50.291330 31692 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:50.237772  3853 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:50.295964 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55118.6us
I20260709 12:25:50.302174 31058 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:25:50.302088 31064 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:50.304548 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.305919 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.238477  3755 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
W20260709 12:25:50.328805 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 112862us
I20260709 12:25:50.287184  3512 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.343137 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58019us
I20260709 12:25:50.298259  3897 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.250015  3875 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.298048  3717 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.299800  3919 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:50.293977  3893 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:50.340134 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.276998  3201 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:25:50.299971  3634 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:50.341324  3691 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:50.367476 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.368616  3932 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:50.370311 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.356928 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.345759  3712 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:50.298126  3928 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.395897 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:50.337651  3813 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:25:50.352844  3711 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.354741  3918 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:50.344153  3759 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:25:50.417448 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.346407  3512 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:50.440733 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 94081.9us
I20260709 12:25:50.352535  3717 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.403142  3712 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:50.358109  3634 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.424163  3813 consensus_queue.cc:1243] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:50.422544  3901 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:50.455237  3717 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.455526  3634 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.374161  3581 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.404155  3853 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.352876  3897 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.461818  3897 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:50.462503 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 936) took 1104 ms. Trace:
I20260709 12:25:50.467665 31138 rpcz_store.cc:276] 0709 12:25:49.357721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.664565 (+306844us) service_pool.cc:224] Handling call
0709 12:25:49.664911 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.982376 (+317465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.982399 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.986961 (+  4562us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.298201 (+311240us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:50.298260 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.298329 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.462280 (+163951us) raft_consensus.cc:1685] finished
0709 12:25:50.462304 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.462443 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":312628,"raft.queue_time_us":320610,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":42635,"thread_start_us":236556,"threads_started":1,"wal-append.queue_time_us":307622}
I20260709 12:25:50.468925  3913 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.446192  3712 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:50.423185  3880 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:25:50.473835  3283 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:50.423331  3879 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:25:50.475997  3928 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:50.445827  3691 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:50.476336  3879 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:50.476863  3691 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:50.477144  3879 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:50.477804  3691 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.376519  3703 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.479030  3691 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.480643  3512 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.481786 31063 raft_consensus.cc:2410] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 12:25:50.463131  3951 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:25:50.486125 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.492300 31058 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:25:50.453675 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:50.482204  3712 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.496163 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.496948 31058 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:50.499727  3963 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.500453  3963 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.502615 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.505975  3963 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.508054 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.508801 31064 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:25:50.441970  3702 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:50.510978 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:50.517213 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:50.518041 31691 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:50.520560 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:50.521347 31692 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:25:50.531635  3712 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:50.534763  3944 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 12:25:50.512419  3901 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:50.541599 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.550151 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:50.516861  3379 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:50.558029 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 212693us
I20260709 12:25:50.558842 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:50.538094  3901 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:50.482679  3907 consensus_queue.cc:1175] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce27a72886964e4287059a1d66bd5b6b" responder_term: 12 status { last_received { term: 12 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 12 index: 7 } }
I20260709 12:25:50.553149  3755 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.561422 31695 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:50.556367  3379 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.485419  3879 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.562620  3875 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.566452  3755 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.793104  3880 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:50.600946  3958 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:50.647567 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.769901  3634 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:50.564041  3591 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.771850  3638 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:50.773337 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:50.775126  3929 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.800462  3880 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.806952  3880 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.801345  3638 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.810595  3638 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.775331  3913 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:50.714162 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.732080 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 173410us
W20260709 12:25:50.792016 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 233322us
I20260709 12:25:50.607244  3964 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:50.827407 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.827845 31058 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:50.788848  3379 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.788074  3712 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:50.802589  3899 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:50.800338  3958 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.802394  3634 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.822633  3929 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.829131 31064 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:50.829254 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:50.832151 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.832914  3899 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:50.830914  3734 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:25:50.831043  3913 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.838351  3634 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.836040 31059 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:50.833801  3712 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.836472  3969 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:25:50.835146  3958 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.829672  3964 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:50.845489  3964 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:50.827121  3802 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.851787 31064 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:50.851969  3967 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:50.852896  3832 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:50.853058  3712 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.853474  3640 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:50.853413  3832 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.851337  3802 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.854038  3640 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.855481  3832 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:50.855737  3802 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.857045  3490 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:50.838891  3913 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.859849 30955 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.835549  3907 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.861467  3893 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:50.861671  3907 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.862560 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.863729 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.863835 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:50.864432 31143 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:50.864859 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:50.865025 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:50.867630  3358 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:50.868019  3358 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:50.868937 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.869144 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.868857  3358 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.869794 30987 raft_consensus.cc:2410] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:25:50.870749 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:50.870801 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:50.871230 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.871361 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:50.872094 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 515234us
I20260709 12:25:50.869671 30991 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:50.872911 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.873409  3808 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:50.869635 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.874547 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:50.872495 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.876060 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.872040 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.878082 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.878513 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.878842 31135 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:50.879488 31139 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:50.882673  3826 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:50.883374 30990 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:50.883476 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:50.883777  3672 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:50.884290 31691 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:50.884687  3782 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:50.882858  3679 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:50.886370 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:50.886499  3679 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:50.883378  3826 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.887467  3679 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.887740  3826 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:50.888660  3679 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:50.890030 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.890862 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:50.890851 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:50.907125 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 55654.1us
I20260709 12:25:50.908802 31059 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:25:50.893282 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.892923 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.903668 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.912958 31064 raft_consensus.cc:2410] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:25:50.864331  3907 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:50.917565 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.856088  3640 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:50.929009 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.934051  3634 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:50.916124  3672 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:50.937808 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.870122  3358 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:50.883639  3894 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:50.890461  3808 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.935063  3913 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.945116 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:50.946550 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.948571 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:50.942039  3672 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:50.950076 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:50.935425  3964 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.950836  3672 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.952162  3672 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:50.952741  3964 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:50.956842  3703 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.957453  3958 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.946873  3640 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:50.958976  3640 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:50.889871  3379 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:50.923484  3929 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:50.980551 31058 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:50.961346 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.965116 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.966826 31060 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:50.982981 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.967376 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.983608 31063 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:50.875506  3557 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:25:50.969348 31137 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:50.974597 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:50.968743 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:50.985049 31138 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:50.985322 30983 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:50.983970 31068 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:50.981871 31141 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:50.987396 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:50.960546 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:50.971045  3846 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.981298  3967 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:25:50.944414  3755 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:50.966005 31064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51942
I20260709 12:25:51.008037 31064 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:50.990792 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:51.008394 31064 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.008702 31136 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:50.991329 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:51.009430 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48974.1us
I20260709 12:25:50.971359  3962 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 12:25:50.995127 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.973073  3581 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:51.003785 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:51.014091 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.014127 30988 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:51.015179 31062 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:25:51.004216 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:50.928519  3802 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.019090 31059 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:51.025998 31691 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
W20260709 12:25:51.020059 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.028008 31106 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:51.042089 31106 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:51.031746 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:50.975495  3808 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.037747 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:51.043771 31059 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:50.994109  3379 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:51.038905 31064 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:51.076839 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45166.9us
I20260709 12:25:51.047052 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:51.077621 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:51.048923 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:51.050912 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:51.083587 30987 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:51.052944 30988 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:51.055416 30992 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:51.062054 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50886.3us
I20260709 12:25:51.026530  3770 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:51.098681 31064 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.064272 30991 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:51.037297  3931 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.075870  3913 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.083765  3782 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:51.013446  3846 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:51.135185  3958 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.139892  3901 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.140566  3901 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.096457  3770 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.142997  3770 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:51.161532  3879 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:51.163559 30983 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:25:51.163926 30983 pending_rounds.cc:85] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 6
I20260709 12:25:51.164143 30983 pending_rounds.cc:107] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:25:51.164382 30983 raft_consensus.cc:2889] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.164672 30983 pending_rounds.cc:107] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:25:51.164893 30983 raft_consensus.cc:2889] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.177176 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:51.027271  3638 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.194410 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:51.195161 31136 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:51.195531 31136 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:51.195931 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2270) took 1185 ms. Trace:
I20260709 12:25:51.196414 31142 rpcz_store.cc:276] 0709 12:25:50.010912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.011046 (+   134us) service_pool.cc:224] Handling call
0709 12:25:50.011371 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.198474 (+187103us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:50.198781 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.890687 (+691906us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b18003e3080. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:25:51.020757 (+130070us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b18003e3080. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a
0709 12:25:51.036894 (+ 16137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.036956 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:51.037020 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.194790 (+157770us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b18003e3080. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87752a6 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419
0709 12:25:51.195666 (+   876us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540001b654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:51.195706 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.195860 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3726,"prepare.queue_time_us":6889,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":12521,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":42643,"spinlock_wait_cycles":553100672,"thread_start_us":59833,"threads_started":1,"wal-append.queue_time_us":59985}
I20260709 12:25:51.205194  3826 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.100311  3891 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:51.206730  3891 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:51.207124  3891 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.208269  3853 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:51.083458  3894 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:25:51.032925  3802 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.113530  3907 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:51.196421 31136 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:51.223380 31136 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.131799  3782 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:51.239614 31698 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:25:51.220031  3814 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:51.133654  3931 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.166493  3913 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.242271  3782 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.242358  3907 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:51.244091  3814 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:51.266654  3901 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.261776  3899 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.274982  3846 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.298731  3808 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.291672  3814 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.304425  3907 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:51.327813  3802 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.336650  3931 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.342839  3913 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.356767  3899 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.427155 31067 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:51.438506 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:51.449970 31144 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.023s	sys 0.063s
W20260709 12:25:51.498090 31144 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.024s	sys 0.063s
W20260709 12:25:51.486716  3634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.024s	sys 0.030s
I20260709 12:25:51.484207  3201 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:25:51.452262 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.014s	sys 0.050s
W20260709 12:25:51.440438 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.438654 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.491616  3679 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.001s
W20260709 12:25:51.509856 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.014s	sys 0.050s
I20260709 12:25:51.502206  3201 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.491593  3638 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.535377  3925 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:51.535939  3925 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:51.476428 30888 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.542804  3983 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:51.543498  3983 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.544003  3983 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:51.522153 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 84146.3us
I20260709 12:25:51.548123 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:51.549459  3929 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:51.549845  3929 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.550179  3929 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:51.455180 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 509355us
W20260709 12:25:51.422003 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 244676us
I20260709 12:25:51.492035  3591 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.566766  3591 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.570355  3931 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.571197  3931 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.574474  3846 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.575614  3846 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.583856  3808 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.383193  3891 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.585019  3808 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.587321  3891 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.586941  3638 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:51.589733 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:51.592036  3900 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:51.592118 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:51.592912  3900 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:51.525215  3201 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:51.600033  3814 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:51.600963  3591 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:51.526235 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 584377us
I20260709 12:25:51.602643  3846 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.603619  3976 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:51.604739  3976 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:51.605149  3976 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:51.606364 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.612640 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:51.613746 31067 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.615099 30991 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:51.612917 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:51.620677 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:51.621866  3897 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:51.622394  3897 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:51.627228 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 79367.3us
I20260709 12:25:51.574133  3931 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.624766  3897 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.636081 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:51.645318 30983 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:51.557704  3925 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.646587 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:51.600040  3900 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:51.649816 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.013s	sys 0.008s
W20260709 12:25:51.651006 31106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.003s	sys 0.011s
I20260709 12:25:51.656447  3966 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.657034 30983 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:51.657199  3966 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:51.659646 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.014s	sys 0.009s
I20260709 12:25:51.649971 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2284) took 1438 ms. Trace:
W20260709 12:25:51.669170 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 542427us
W20260709 12:25:51.674782 30990 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.009s	sys 0.018s
I20260709 12:25:51.680052 31067 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:51.685026 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:51.701895 30983 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:51.702399  3966 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.702963  3931 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.677253 31136 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:51.688659 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:51.680689 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:51.671906  3742 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:51.651433  3782 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:51.678309 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 610924us
W20260709 12:25:51.685168 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 643096us
I20260709 12:25:51.691061 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:51.714989 30987 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.685580  3899 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:51.726688  3802 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.727474  3802 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.730916  3879 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:51.731958  3985 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:25:51.731657  3879 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.733873  3879 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.681304  3976 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.734372  3802 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.735328  3976 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:51.738711 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:51.739809 31136 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
W20260709 12:25:51.741142 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 60687us
W20260709 12:25:51.748857 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.750330 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:51.688047 30888 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.753840 31139 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:51.768647 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:51.768769 31068 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:51.769322 31060 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
W20260709 12:25:51.778446  3770 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.009s
I20260709 12:25:51.662277 31144 rpcz_store.cc:276] 0709 12:25:50.209125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.263452 (+ 54327us) service_pool.cc:224] Handling call
0709 12:25:50.263704 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.263894 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:50.264426 (+   532us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.498917 (+1234491us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.630168 (+131251us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.632009 (+  1841us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:51.641155 (+  9146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.646919 (+  5764us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.647514 (+   595us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":586330,"prepare.queue_time_us":476353,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":79,"raft.queue_time_us":596502,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":318,"thread_start_us":765951,"threads_started":2,"wal-append.queue_time_us":149346}
I20260709 12:25:51.779115  3853 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.778976  3770 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:51.779771  3853 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.759774  3879 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:51.781638  3982 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:51.782335  3879 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:51.783555  3893 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:51.784729  3982 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:51.784260  3893 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.786182  3982 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:51.786710  3982 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:51.787012 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:51.787356  3893 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.784905 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:51.788304 31063 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:51.788115  3982 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:51.789446 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:51.789877  3982 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:51.794653 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:51.795536 31058 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:25:51.798521 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 170010us
I20260709 12:25:51.799394 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:51.801640  3711 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:51.801777 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:51.802608  3925 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:51.803227  3711 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:51.803416 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:51.804332 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:51.805604 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:51.806481  3846 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:51.807195 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:51.784648  3879 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.808022  3770 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.810483 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:51.811141 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:51.811762  3893 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.811975 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:51.812660 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:51.760596  3808 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.815193  3591 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.815932  3891 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:51.816587 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:51.817991 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:51.818923 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:51.819051 30888 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.772882  3899 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:51.782101  3853 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.821115 31693 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:51.814750 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:51.793870  3897 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.851437  3901 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:51.765684  3931 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:51.864212  3846 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:51.871302 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:51.871536 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:51.871712  3846 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.874722 31137 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:51.851740  3679 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.876075  3679 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.853806  3879 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.890614 31692 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:25:51.896138  3703 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.939s	user 0.022s	sys 0.004s
I20260709 12:25:51.902820 31064 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:51.879035  3742 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:25:51.909381  3986 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:25:51.912340  3853 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:51.913339 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.914454 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:51.915391 31143 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
W20260709 12:25:51.916975  3581 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.889s	user 0.009s	sys 0.007s
I20260709 12:25:51.917429  3581 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:51.917580  3897 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:51.918571  3358 consensus_meta.cc:287] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.972s	user 0.007s	sys 0.015s
I20260709 12:25:51.920486  3931 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:51.921105  3931 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:51.921487  3931 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.921859  3931 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:51.864624  3591 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.924355  3591 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.900277  3893 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.928709  3893 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.931263 31689 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:25:51.865456  3901 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.921345 30888 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 8 to 11, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 11 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.849522  3201 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.913033  3853 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:51.932271  3672 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.980s	user 0.012s	sys 0.011s
W20260709 12:25:51.958758 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 87636.8us
I20260709 12:25:51.965996 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:51.945653  3901 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.947037  3679 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.980844  3913 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:51.981683 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:51.981621  3913 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.984336  3913 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:51.965148  3814 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:51.985136 31138 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:25:51.998670 31138 pending_rounds.cc:85] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 9
I20260709 12:25:51.992987 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:51.992993 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47085.3us
I20260709 12:25:52.006799 31693 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:25:52.007678  3782 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:52.009610 31060 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 12:25:52.010079 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.015681 30987 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
W20260709 12:25:52.016104 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 214454us
I20260709 12:25:51.987676 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:51.998929 31138 pending_rounds.cc:107] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:25:51.976296  3581 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:51.964356  3891 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:52.034264 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61571.1us
I20260709 12:25:51.965757  3893 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:52.001178 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.020242 31138 raft_consensus.cc:2889] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.037494 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.038204 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:52.038496 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.038728 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.058320 31142 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.040166 30888 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 6 to 10, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:52.041883 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.042481  3996 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:52.045408 30985 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:52.045742 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:51.990890  3802 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:51.993188  3358 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:52.056310 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:51.985024  4002 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.060911 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:52.034883  3985 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:25:52.065496 30984 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:51.985664  3814 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:51.987808  3879 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.068885 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:52.072851 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.077248 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.080399 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.081033 31063 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:52.081372  3201 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.079489 31137 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:52.085005 31065 raft_consensus.cc:2368] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:25:52.085891 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.082509 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.063112  3802 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:52.095579 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49631.9us
I20260709 12:25:52.095971 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.093664 31139 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:52.100736 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.096304 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2658) took 1026 ms. Trace:
I20260709 12:25:52.104344 30986 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.082986  3358 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:52.087852  3913 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:52.126960  3932 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:52.096342  3996 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:52.134939 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 89294.2us
I20260709 12:25:52.135473 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:52.140157 30887 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 320) took 30975 ms (client timeout 15000 ms). Trace:
W20260709 12:25:52.140409  3983 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.003s
I20260709 12:25:52.141399 30888 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 9 to 13, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 13 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:52.143162 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.075127  3893 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.090539  3581 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.129873  3802 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.149755 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.152318 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:52.153146 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 156449us
I20260709 12:25:52.154654  3814 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.144903  3846 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.180470 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:52.155429 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.159420  3879 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.160673  3891 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:52.182431 31689 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:25:52.182932 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.183477 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:52.096408  3982 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:52.196401  3993 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.161s	user 0.002s	sys 0.000s
I20260709 12:25:52.164229 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.104642  4002 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:52.162591 30887 rpcz_store.cc:269] 0709 12:25:21.164738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.206338 (+ 41600us) service_pool.cc:224] Handling call
0709 12:25:21.553949 (+347611us) spinlock_profiling.cc:243] Waited 5.41 ms on lock 0x7b5400002210. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee87700b2 00007f2ef16e9759 00007f2ef1681a76 00007f2ef1652d58 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc
0709 12:25:52.139889 (+30585940us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:43.690666 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:43.691579 (+   913us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:43.691649 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:25:43.691677 (+    28us) tablet.cc:662] Decoding operations
0709 12:25:43.694196 (+  2519us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:43.694350 (+   154us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:43.694391 (+    41us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:25:44.052829 (+358438us) tablet.cc:701] Row locks acquired
0709 12:25:44.052899 (+    70us) write_op.cc:260] PREPARE: finished
0709 12:25:44.053112 (+   213us) write_op.cc:270] Start()
0709 12:25:44.053313 (+   201us) write_op.cc:276] Timestamp: P: 1783599944053032 usec, L: 0
0709 12:25:44.053358 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 12:25:44.156766 (+103408us) log.cc:844] Serialized 9691 byte log entry
0709 12:25:44.693683 (+536917us) op_driver.cc:464] REPLICATION: finished
0709 12:25:45.177993 (+484310us) write_op.cc:301] APPLY: starting
0709 12:25:45.210643 (+ 32650us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:46.165649 (+955006us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:46.165783 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:46.774363 (+608580us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:46.774818 (+   455us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 12:25:46.774863 (+    45us) write_op.cc:312] APPLY: finished
0709 12:25:46.775508 (+   645us) log.cc:844] Serialized 220 byte log entry
0709 12:25:47.483579 (+708071us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2ee878a0ac 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b
0709 12:25:48.577753 (+1094174us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:48.596481 (+ 18728us) write_op.cc:454] Released schema lock
0709 12:25:48.689061 (+ 92580us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:48.778676 (+ 89615us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:48.782435 (+  3759us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":297193,"reported_tablets":47,"spinlock_wait_cycles":12447616,"child_traces":[["op",{"apply.queue_time_us":383692,"apply.run_cpu_time_us":101103,"apply.run_wall_time_us":3626863,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":151764,"prepare.run_cpu_time_us":48019,"prepare.run_wall_time_us":672113,"raft.queue_time_us":76264,"raft.run_cpu_time_us":12783,"raft.run_wall_time_us":344562,"replication_time_us":587131,"spinlock_wait_cycles":45375744,"thread_start_us":2172189,"threads_started":5,"wal-append.queue_time_us":1806815,"wal-append.run_cpu_time_us":17705,"wal-append.run_wall_time_us":2227179}]]}
I20260709 12:25:52.168823  3899 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.169695  3853 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:52.170153  3990 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:25:52.199822  3893 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.169952  3983 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.102639 30990 rpcz_store.cc:276] 0709 12:25:51.063068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.065242 (+  2174us) service_pool.cc:224] Handling call
0709 12:25:51.065628 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:51.066070 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.066109 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:51.066792 (+   683us) log.cc:844] Serialized 52 byte log entry
0709 12:25:51.767394 (+700602us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:51.968319 (+200925us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.065533 (+ 97214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.071838 (+  6305us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.071875 (+    37us) raft_consensus.cc:1685] finished
0709 12:25:52.071888 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.089523 (+ 17635us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35646,"prepare.queue_time_us":129898,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":56140,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":27352,"thread_start_us":751384,"threads_started":3,"wal-append.queue_time_us":543802}
I20260709 12:25:52.174580 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.202845 31691 raft_consensus.cc:2368] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:52.174120  3679 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.203802  3983 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.203761  3679 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.205909  3679 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.181269 31064 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:52.184119 31696 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:52.187588 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.211012 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.180169  3201 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:52.161474 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.217933 31106 leader_election.cc:400] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:25:52.220161 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.220512  3982 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:52.220928 31058 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:52.221271  3982 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:52.221782 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.222476 31068 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:52.222674  3982 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:52.223883  3891 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:52.224251  3891 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:52.225200  3891 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.228035  4007 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:52.188766  3846 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.232143  3846 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.244910 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:52.245584 31693 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:52.246824 31696 raft_consensus.cc:2410] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:25:52.250129  3976 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.001s
I20260709 12:25:52.254343  3581 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:52.248018 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 112689us
I20260709 12:25:52.249047  3802 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.263289  3358 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.266510 30888 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:52.269979 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:52.273416 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.274109 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.274812 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.275425  3891 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.276470 31695 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:52.276793 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:52.287977  3638 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.302340 31696 raft_consensus.cc:2368] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:52.303854  4002 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.305764  3996 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.307857  3638 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:52.310496  3929 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.760s	user 0.011s	sys 0.001s
I20260709 12:25:52.314270  3929 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.315917 30991 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:52.309996  3638 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:52.319707  3929 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.295260 31106 leader_election.cc:403] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:25:52.321866 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 104121us
I20260709 12:25:52.322027  3891 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.323603 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:52.324302  3808 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:52.325793  3893 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:52.325971 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.326272  3893 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:52.326686  3893 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.328215  3808 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:52.328625  3808 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:52.296458 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 82708.1us
I20260709 12:25:52.329680  3808 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.332693  3808 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:52.338011 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.343451  3901 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.345722  3901 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:52.347506  3925 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.517s	user 0.009s	sys 0.000s
I20260709 12:25:52.348619 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.350297  3899 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.352272 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.358455  3201 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:52.358973  3899 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.359599  3899 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.359946 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.361653  3899 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.362711  3894 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:52.365074  3891 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:25:52.365228  3894 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:52.365828  3894 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.368078  3894 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:52.379753 30888 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 1 to 6, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 6 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:25:52.385428  3891 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.386525  3891 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.388239 30986 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:52.388424  3891 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:52.291642 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:52.395119 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 103595us
I20260709 12:25:52.320832  3931 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.398909  3931 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.410249 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.411101 31064 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:52.300638 31142 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:52.424175 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.431903 31693 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:52.432930  3814 raft_consensus.cc:2764] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:52.425912  3638 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:52.439425 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 259032us
I20260709 12:25:52.426373  3201 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:52.441057 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 276906us
I20260709 12:25:52.434551  3964 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.440943  4006 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.444890  4008 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:52.438529  3638 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.450416  4008 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:52.450583  3638 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.450937  4008 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:52.454586 30955 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:52.442325  3891 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.455597  3891 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.458447  3996 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:52.458685 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.459316 31059 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:52.459498 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.449594  4006 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.460103 31064 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 7.
I20260709 12:25:52.461580  3891 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.461680 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:52.445037  3853 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.462357 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:52.464542 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 79010.9us
I20260709 12:25:52.465503 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:52.466754 31067 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.468505 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.469594 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:52.471316 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:52.472448 31065 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:52.470798 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.476675 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.478022  4006 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.466580  3853 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.479080  3853 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.481046 31061 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:52.481971 31068 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:25:52.482333 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 122510us
W20260709 12:25:52.478307 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.483229 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.483784 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:52.484393 31060 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.484543  3782 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:52.485390  3201 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:52.485224  3782 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.485503 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:52.486955 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:52.487695 31068 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:52.487957  3814 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.488065 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:52.488411  3814 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:52.488862  3814 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.489267 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:52.489442  3358 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:52.490610  3962 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:52.491019  3962 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.491395  3962 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.492962  3634 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:52.493559 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:52.501988 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:52.502959  3983 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.503739  3983 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.504745  3925 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:52.466457  3894 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.485224 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.506096  4005 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.506392  3894 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.506886  4005 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.507433  3770 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:52.508967  3894 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.509779  3929 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.512399  3929 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.515575 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.511427  3846 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.515702 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:52.510767 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45374.9us
I20260709 12:25:52.510535 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:52.528771 31067 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:52.530459 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.539992 30986 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:52.530678 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.534040 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.535398 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:52.536240 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:52.536720 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.537667 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.538048 30985 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:25:52.539076 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.544955 31064 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:52.538723 30990 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: [15.7-15.7]
W20260709 12:25:52.483197  4012 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:52.555852 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.515039  3782 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:52.545373 30984 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:52.530187 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.564061 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.565179 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.566413 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.567203 31065 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:52.563913 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.568885 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.569945 31068 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:52.571267 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.564728 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.564584 30992 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:52.576833 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.534189  3996 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.542461  3983 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.578639 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:52.581027 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.582617 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.533113  3913 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.533496  3770 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:52.534369  3984 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.558724  3931 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.562734  4002 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:52.589118 31695 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:52.578092  3925 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:52.534636  3899 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:25:52.530750  3581 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:25:52.601194 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:52.604506 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.597483  3925 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.613013 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:52.625425 31691 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:52.538975  3814 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.587159  4006 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:52.624187 30955 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.624864 31692 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:52.587463  3894 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.587617  3679 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:52.599258  4002 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.532480  3962 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.561374  3893 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.632553 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.633100 31142 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:52.557396  3964 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:25:52.599962  3770 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.637530 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:52.626395  3802 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:52.632728  3891 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.649849 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.636145  3679 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.598944  3983 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:25:52.651934 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.647775 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.632589  4006 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:52.655825 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.652513  3770 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.644547  4002 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.599855  3913 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.660198 31692 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:25:52.656517 31135 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:52.665529 30992 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:52.582505  4008 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.662732 31142 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:52.634663  3982 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:52.663753 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:52.633435  3984 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.657088  3891 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.636540  3996 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.646315  3931 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.672937 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.644641  3894 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.660392  3679 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.687944 31691 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:25:52.674319 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.663340  3802 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:52.665203  4006 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.691969 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.665203  4007 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:52.666291  3932 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.670751  4023 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.675582  3969 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.714360 31140 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.714650  3969 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.715801  3962 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.716771 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:52.716933  3969 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:52.717321 31065 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.717473  3931 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.672685  3982 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.720103  3982 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.721731  3984 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.723393  3996 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:52.727044  4017 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:25:52.714540  3929 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.704041 30987 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:52.703728 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:52.757922 30992 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:52.688063  3891 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.763034 31060 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:52.709077  4020 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:25:52.709368  3814 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.710682  3893 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.675048  3907 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:52.729964  3742 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:52.762895 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.708881 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.792264 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.709689  3932 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.752805 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:52.775399 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68839.2us
I20260709 12:25:52.798805 31140 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:52.799088 31142 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:25:52.799697 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:52.786892 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.793355 31141 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:25:52.694425  3853 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:52.804406 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 101837us
I20260709 12:25:52.805383 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.794480 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.806638 31135 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:52.795071 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.795526 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:52.792202  3742 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:25:52.807647 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 54939.1us
I20260709 12:25:52.707965  4005 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.834514 30991 raft_consensus.cc:2368] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:52.834947 31059 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:52.709784  3802 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.842765 31058 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:52.846467 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.790733  3964 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.847813 31060 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
W20260709 12:25:52.850018 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44922.8us
I20260709 12:25:52.792007  3907 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.855450 31065 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:52.856446 31064 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:52.857201 31695 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:52.859164 31142 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:25:52.859632 31142 pending_rounds.cc:85] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 10
I20260709 12:25:52.859905 31142 pending_rounds.cc:107] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:25:52.860195 31142 raft_consensus.cc:2889] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.861177 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:52.861469 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.800096  3591 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:52.805702  3984 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.866109 31109 leader_election.cc:400] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:52.866844 31136 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:52.866990 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.870052 31691 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:52.874557 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.844451  3996 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:52.868822 31144 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:25:52.876472 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:52.878957 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.880775 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.718170  4023 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.868580 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.881384 31696 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:52.851490  3853 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.854844  3907 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.893052 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:52.896548  4006 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.903286  3932 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.874881  3982 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:52.886998  3998 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:52.919781 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.878978  3996 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.887331  4027 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:52.887760 31109 leader_election.cc:400] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:52.920830 31109 leader_election.cc:403] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:25:52.883100  3901 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.907958 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.909293 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:52.909816  3964 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:52.952855  3901 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:52.966444  3901 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.865216 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.968983 31143 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.969141 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.969741 31144 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:52.974790 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:52.975569 31689 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:52.976478 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 67267.9us
I20260709 12:25:52.977242 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.977365 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.977969 31694 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:52.978080 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.978928  4038 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:52.978947 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:52.979409  4038 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.979825  4038 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:52.980557 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:52.981231  4041 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.981684  4041 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.982201 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.904016 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.983491  4024 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:52.983736 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:52.983775 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 79828.7us
I20260709 12:25:52.985082 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.985944  4029 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.986282 31664 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:52.986423  4029 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:52.990166 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:52.956547 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68935.3us
I20260709 12:25:52.977391  4039 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.990808 31141 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:52.991084 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:52.946362  4032 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.990134 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:52.923604 31138 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:25:52.991837  4032 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:52.992589  3358 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:52.992969  3358 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.993295  3358 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:52.892009 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:52.994714 31136 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:52.958925  3679 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.929553  3742 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:52.996824  3742 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.998518 30989 raft_consensus.cc:2368] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:52.994047  4032 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:52.913867 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 252270us
I20260709 12:25:52.964087  4008 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.964907  3913 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:53.004041  3679 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.982375  3932 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:52.982540  4037 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:53.006397 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:53.009697  3913 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:53.009727 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:53.010335  3913 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:53.011132 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:53.011247  3913 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:53.011770  4023 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:53.012151 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:53.012719 31065 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:53.014006  3638 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.002s
I20260709 12:25:53.014662 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.982028  4041 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.984951  3853 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:53.028990 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.990401  4029 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.878810  4005 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:53.030700  4005 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.032982 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.033196  4005 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.927464  3996 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:52.953505  3894 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:52.985980  3962 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:53.035056 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:52.929734 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.988358  3770 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.037858  3770 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:52.976648 31690 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:53.011736 31061 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.962023 31697 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:53.018657 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:52.958006 31135 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:53.020150 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:53.041637 31065 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 8.
I20260709 12:25:53.041978 31060 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:53.022118 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:53.042438 31058 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.028766  3672 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
W20260709 12:25:53.066818 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 111901us
I20260709 12:25:53.069734 31065 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:53.068011 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.076324 31143 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:53.078557 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:53.079334 31691 raft_consensus.cc:2410] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:52.868243  4007 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:53.080530 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:53.088158 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 97171.4us
I20260709 12:25:53.088842 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:53.083024 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:53.078266 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.090186 31689 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:53.094566 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:53.095101 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:53.095733 30986 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:53.014536  3638 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:53.101024 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:53.125512 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:53.124661 31068 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:53.039973  3982 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:53.098393 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:53.133001  3782 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:53.143580 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:53.144351 31141 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.146080 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:53.146749 31137 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:53.150435  4025 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:53.151433  3984 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:53.139544 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:53.121985  3581 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.177258  3581 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.179505  3581 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:53.173732 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:53.180755 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:53.099545  4023 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:53.087020  3932 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.191221 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:53.174494 31693 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:53.175416 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:53.101146  3894 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:53.187548 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:53.189357  3591 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.189599 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:53.191768  3932 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.190656 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:53.192059 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.184649 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:53.200809 30991 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:53.076143  4038 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.200194 31689 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:53.201956 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:53.193197 31059 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:53.189926  4007 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.196560 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.198359 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:53.202106 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 62653.8us
I20260709 12:25:53.134716  3893 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:53.184514 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:53.097594  4006 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 8 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:53.204806 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:53.209107 31140 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:25:53.040422  3891 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:53.206624 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:53.191275  4023 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:53.152716  4020 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:53.209735 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:53.087872  4002 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.211519 31142 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:53.206784 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:53.211871 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:53.213054 31061 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:53.207472 31144 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:25:53.205559  3782 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:53.289465 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 79795.2us
I20260709 12:25:53.211390  3925 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:53.303715 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:53.279698 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 63016us
I20260709 12:25:53.214895  4005 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:53.322088 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:53.138922  3672 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:25:53.324361 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:53.324965 31142 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.218595  4020 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:53.200562  3907 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:53.201504  3591 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.332513 30985 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:53.337677 31140 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:53.218681  4021 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
W20260709 12:25:53.338632 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 198175us
I20260709 12:25:53.338652 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:53.340837  4006 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:53.305770  4043 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:53.373517 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:53.382097 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48856.8us
W20260709 12:25:53.386161 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 178008us
I20260709 12:25:53.324219  4005 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:53.387859 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 181132us
I20260709 12:25:53.386205 30958 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:53.390650 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:53.393471 30986 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:53.323980  3925 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.338728  4040 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:53.331879  3913 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:25:53.428361 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:53.394033 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:53.424957 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:53.438726 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44821.7us
I20260709 12:25:53.472087  3591 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:53.472615 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48502.2us
I20260709 12:25:53.473747 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:53.484091 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:53.486765 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.487986 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.489338  4018 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.216s
I20260709 12:25:53.424682  3925 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:53.502985  4046 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
W20260709 12:25:53.505254  4042 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
I20260709 12:25:53.508472 30992 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:25:53.512032 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 84910.8us
I20260709 12:25:53.517167 31695 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:25:53.521293 31138 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:53.396522  4005 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:53.391973  4043 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:53.399307  4032 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:53.627054 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54010.7us
I20260709 12:25:53.381102  3782 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.427721  3679 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:25:53.577780 31058 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.000s
I20260709 12:25:53.661795 31058 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:53.427956  4040 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:53.615356  4029 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.000s
I20260709 12:25:53.566233  4032 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:53.647255  4041 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.000s
I20260709 12:25:53.383306  3982 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:53.674503 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:53.683707 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 290332us
I20260709 12:25:53.638696  3638 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:53.685704 31144 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:53.657833  3853 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:53.575104  4043 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.666329  3679 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:53.678542  4040 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:53.679620  4032 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:53.540504  4018 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.159s
W20260709 12:25:53.702883 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 229196us
I20260709 12:25:53.703593 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:53.704639 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:53.705611 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:53.714924 31141 consensus_meta.cc:287] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.000s
I20260709 12:25:53.715724 31141 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:53.716493 31140 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [13.11-14.12]
I20260709 12:25:53.719208 31142 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:53.687078  3964 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:53.729988 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 53189.9us
I20260709 12:25:53.677699  4029 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.680779  3581 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:25:53.686717  4041 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.695729  3679 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:53.507719  4023 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:53.735275 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 50443.6us
I20260709 12:25:53.696841  3853 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:53.696485  3932 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:53.696916  4038 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:53.696435  4051 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:53.738876  3358 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.745s	user 0.001s	sys 0.010s
I20260709 12:25:53.770967 30987 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:53.740052  3891 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:53.738595  3964 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:53.781890 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:53.743714  4002 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:53.791126 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 392301us
I20260709 12:25:53.792095 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.795800 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:53.836473 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:53.767784  4051 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:53.776616  3591 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:53.837301 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:53.774394  4044 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:53.895151 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57941.4us
I20260709 12:25:53.791060 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:53.896030 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:53.901240 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.907065 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:53.907641 31697 raft_consensus.cc:2376] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12. Re-sending same reply.
W20260709 12:25:53.907789 30991 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.706s	user 0.006s	sys 0.003s
I20260709 12:25:53.908233 30991 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:53.908653 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:53.845595  4048 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.458s	user 0.001s	sys 0.000s
I20260709 12:25:53.910557 31142 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:53.837224  4038 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:53.836122  3984 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:53.782784  3358 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:53.901089  4052 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:25:53.793109  3907 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:53.913007 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.921541 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:53.952095 30991 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:53.945750 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:53.953028 31691 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:53.945467  4044 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:53.954298 31138 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:53.955996  4044 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:53.966794  3907 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:53.967015  3782 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:53.967484  3907 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:53.968027  3907 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:53.971987  3358 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:53.973016  4049 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 12:25:53.924937  3853 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:53.915721 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:53.946682 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:53.980185 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 64558.7us
I20260709 12:25:53.948935 31693 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 11.
I20260709 12:25:53.836088  4059 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:53.884011  4051 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.990084 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:53.923175  4047 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.396s	user 0.001s	sys 0.000s
W20260709 12:25:53.922693  3901 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.953s	user 0.009s	sys 0.002s
I20260709 12:25:53.911332  3932 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:53.913777  4038 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:53.914089  3672 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:53.897986  4058 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:54.010757 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.005664 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.012065 31139 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:25:54.007294 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 254005us
I20260709 12:25:53.970083  3907 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:53.992285  3638 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:54.031993  3638 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:54.033192  3638 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.014309 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.030386  3802 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:54.034310  3770 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:54.031312 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1131) took 1449 ms. Trace:
I20260709 12:25:54.009821 30991 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:54.034355 31138 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.012336  4038 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.009052 31142 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:54.035344 31698 rpcz_store.cc:276] 0709 12:25:52.582137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.590696 (+  8559us) service_pool.cc:224] Handling call
0709 12:25:52.627772 (+ 37076us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.799528 (+171756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.799561 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.836757 (+ 37196us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.192637 (+355880us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:53.192678 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.192733 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.031009 (+838276us) raft_consensus.cc:1685] finished
0709 12:25:54.031038 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.031217 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6353,"raft.queue_time_us":44150,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":436544,"threads_started":2,"wal-append.queue_time_us":455127}
I20260709 12:25:54.083889 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:53.988562  4059 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.006481  4021 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:54.091670 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 386091us
I20260709 12:25:54.092549 31109 leader_election.cc:400] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:54.092882 31109 leader_election.cc:403] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:54.093848 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:25:54.097415 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.046191 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:54.098153 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 52047.5us
I20260709 12:25:54.098766 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.099896  4059 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:54.105073 31109 leader_election.cc:400] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:54.034767  3802 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.108386 31109 leader_election.cc:403] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:54.109788 31138 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:54.034785  3770 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.083060  3932 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:54.107264  3742 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:54.118590 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:53.988723  3984 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.124873 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2510) took 1106 ms. Trace:
I20260709 12:25:54.012940  4040 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.125469 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:54.129522 31142 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:54.131719 31698 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:54.107573  3891 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:54.133267 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.039788  3901 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.126745 31064 rpcz_store.cc:276] 0709 12:25:53.008157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.038289 (+ 30132us) service_pool.cc:224] Handling call
0709 12:25:53.038585 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.038937 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.038969 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.039410 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.092576 (+ 53166us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:53.092631 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.092720 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.113967 (+1021247us) raft_consensus.cc:1685] finished
0709 12:25:54.113988 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.114115 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":910,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":33791,"thread_start_us":52754,"threads_started":1,"wal-append.queue_time_us":237245}
I20260709 12:25:54.126296  3742 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.113466  4021 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:54.142051 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 239153us
I20260709 12:25:54.135149  3358 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:54.123534  3770 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:54.126035  3581 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:25:54.142545 31694 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:54.132625  4029 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.141135  4044 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:54.135408  4002 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.081022  3964 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.154619 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:54.146103  3742 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:54.144291  4007 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:25:54.156625 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57914.6us
I20260709 12:25:54.160462 30991 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:54.156658  3925 consensus_meta.cc:287] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.002s
I20260709 12:25:54.159401 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:54.162580  4007 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.168061  4007 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.144548  4057 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:25:54.169643  4044 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:25:54.170991 31662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.007s	sys 0.010s
W20260709 12:25:54.171300 31662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.007s	sys 0.010s
I20260709 12:25:54.175235  4030 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.175736  4030 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.152276  4068 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:54.182996 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 64649.6us
I20260709 12:25:54.184249 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:54.184864 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:54.185657 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:25:54.188174  4055 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 12:25:54.163866  4064 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:54.189412  4064 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.189054  4051 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.189823  4064 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:54.190577  4056 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:25:54.190387  4021 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.192210 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.194716 30991 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:54.184155 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:54.198886 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.199719  4076 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:54.199743  3891 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.201540 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:54.202631  3891 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:54.203292  3891 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:54.177493  4002 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.196770  3581 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:54.205782  3996 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.172s	user 0.012s	sys 0.000s
I20260709 12:25:54.206442  4002 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.206230  3996 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:54.207363 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.036s	sys 0.028s
W20260709 12:25:54.207680 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.036s	sys 0.028s
I20260709 12:25:54.208361  3996 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.209283  3679 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.181137  3925 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.210700  3802 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.211455  3802 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.212200  3925 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:54.211992  3802 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.212488  3591 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.212965  3591 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:54.214442 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 703078us
I20260709 12:25:54.214797 31695 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:54.218135 31058 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:54.222657 30989 raft_consensus.cc:2368] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:25:54.181674  4029 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:54.226377 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:54.232314 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:25:54.226425 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 61625.7us
I20260709 12:25:54.232796 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.238449 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:54.233870 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.242815 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.243530 31693 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:54.198376  4068 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.223294  3984 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:54.249356 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 47935.2us
I20260709 12:25:54.252279 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.253417  4040 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.258378 31693 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:25:54.260581 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.261204 31059 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.262859 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:54.266027 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.238068  3358 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:54.267640 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.269711  4071 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.274432  4067 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:54.275583 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.276158 31143 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.276991  3679 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.248885  4076 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:54.278371  4066 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:54.278990  4076 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.280232 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:54.196924  3932 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.284004 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.279145  3742 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.287015 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:54.204376  3891 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.251325  4029 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:54.255254  4007 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:54.277956  3591 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.303007  3742 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.303066  4029 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.303853  3591 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.294358  4079 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:25:54.282564  4030 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.260100  3802 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.306553  4081 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:54.306819  4051 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:54.307128  4079 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:54.290411  4067 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:54.307514  4079 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:54.260668  4040 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.247803  4057 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:25:54.299507 31059 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
W20260709 12:25:54.288702 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 54905.6us
I20260709 12:25:54.301873  3679 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:54.231794  4043 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:54.300705  4032 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 12:25:54.273923  4071 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.324519  4079 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.325387  3742 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.326279 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.315326 31143 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:54.334491 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:25:54.333658 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 77266.2us
I20260709 12:25:54.307843  4067 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:54.337333  3742 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.338430  4081 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.338624 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:54.339303 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.339390  4079 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.339977 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:54.340566 31064 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:54.353868 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:54.285512  3358 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.336741 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:54.354714 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:54.282032  4064 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.355684 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.361104  3742 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.361575  3679 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:54.364507 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.367007 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.367926 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.387794 31062 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:54.368185 31059 raft_consensus.cc:2410] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:25:54.368728 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.369431  3846 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:54.370669  3770 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.334761  4032 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.373402  3962 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:54.290095  4066 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:25:54.388520 31067 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:54.340085  4068 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.395933 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.389338 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:54.326021  4040 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.344038  4067 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:54.377792  4044 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:54.389343  4057 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:54.400049  3814 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:25:54.396559  3802 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.404279 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.407970 31694 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:54.406668 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 994) took 1816 ms. Trace:
I20260709 12:25:54.395212  3846 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:54.413775  4043 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.397235  3679 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:54.414407  4066 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:54.416298  4040 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.417780  4071 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.423255 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:54.424000 31143 raft_consensus.cc:2410] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:25:54.401078  3962 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:54.436724  4025 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.437158  4025 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.422418  3808 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.438088  3808 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.442209 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:54.441175  3808 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.444388  3358 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:54.444193  4020 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.445470  3846 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.446485  3846 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.411806  4044 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.450425  4044 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.437099  3962 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.451928  3962 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.452803  4070 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:54.453336  4070 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.454427  3964 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.455024  4062 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:54.454957  3964 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.455442  4062 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:25:54.456444  4005 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.825s	user 0.003s	sys 0.008s
I20260709 12:25:54.456300  4062 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.457180  3964 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.457474  4062 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.457702  3964 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:54.434545 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.457861  4037 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.458370 31067 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:54.458436  4037 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.459321 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.405613  3770 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.407562  4032 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.429807  3679 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.422966 30983 rpcz_store.cc:276] 0709 12:25:52.590546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.591159 (+   613us) service_pool.cc:224] Handling call
0709 12:25:52.613935 (+ 22776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.927165 (+313230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.957448 (+ 30283us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.096512 (+139064us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.339722 (+243210us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:53.339762 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.406250 (+1066488us) spinlock_profiling.cc:243] Waited 92.5 ms on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:54.406307 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.406360 (+    53us) raft_consensus.cc:1685] finished
0709 12:25:54.406375 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.406587 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31512,"raft.queue_time_us":85637,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":9075,"spinlock_wait_cycles":212719872,"thread_start_us":371668,"threads_started":2,"wal-append.queue_time_us":509640}
I20260709 12:25:54.468221 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:54.456863  4005 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.422274  4076 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:54.449723  4038 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.460825  4057 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:54.496076 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.492744 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:54.484284 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:54.503615 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 999) took 1433 ms. Trace:
W20260709 12:25:54.540952 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48293.7us
I20260709 12:25:54.487576 31058 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:54.498682 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2322) took 1315 ms. Trace:
I20260709 12:25:54.542681 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.434549  4066 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:54.480774 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.490388  4037 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.488862  4070 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.492486 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.483709 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:54.553220  4059 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.553668  4038 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.554436 31142 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:54.554888  4064 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:54.555133 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.502437  3814 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:54.487712  4067 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.556010 31689 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:54.556052  3814 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.557416  3814 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.557567  4067 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.512506 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:54.558490 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 46070.9us
I20260709 12:25:54.560702  4064 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:54.561190  4064 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:54.562011  4064 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.567134 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.567200  4044 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.567674 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2331) took 1191 ms. Trace:
I20260709 12:25:54.567837  4005 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.568362 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.569389 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.550565  3808 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.540117  4076 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:54.575295 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.575320  4076 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.576488 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.577579 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.578526 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.579037 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2419) took 1538 ms. Trace:
I20260709 12:25:54.579427 30987 rpcz_store.cc:276] 0709 12:25:53.040157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.040421 (+   264us) service_pool.cc:224] Handling call
0709 12:25:53.086015 (+ 45594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.772217 (+686202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.578832 (+806615us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b540014dcd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:54.578985 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554119552}
I20260709 12:25:54.580541 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.580688  3901 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.581938 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2426) took 1406 ms. Trace:
I20260709 12:25:54.582320 30985 rpcz_store.cc:276] 0709 12:25:53.175438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.211630 (+ 36192us) service_pool.cc:224] Handling call
0709 12:25:54.581742 (+1370112us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b5400024ed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:54.581874 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9476992}
I20260709 12:25:54.583437 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:54.568090 31140 rpcz_store.cc:276] 0709 12:25:53.376424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.475290 (+ 98866us) service_pool.cc:224] Handling call
0709 12:25:53.716252 (+240962us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:53.716825 (+   573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.567506 (+850681us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001ea0d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:54.567619 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553429888}
I20260709 12:25:54.585640 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.590818  4057 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.593950 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.594758 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.595556 31060 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:54.596323 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.597677 31060 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:54.597368  4066 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.542433 31137 rpcz_store.cc:276] 0709 12:25:53.183225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.183370 (+   145us) service_pool.cc:224] Handling call
0709 12:25:53.187212 (+  3842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.201499 (+ 14287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.201526 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:53.201564 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:53.201598 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.498469 (+1296871us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b54001eaad4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:54.498512 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.498627 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32398720}
I20260709 12:25:54.508486 31135 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:54.586179 31140 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:54.603647 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.594769 31694 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:54.541210 30992 rpcz_store.cc:276] 0709 12:25:53.070135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.070512 (+   377us) service_pool.cc:224] Handling call
0709 12:25:54.503368 (+1432856us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b540014ebd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:54.503523 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762486144}
I20260709 12:25:54.507627 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2329) took 1192 ms. Trace:
W20260709 12:25:54.549052 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 79220.3us
I20260709 12:25:54.545735 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:54.549813 30986 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:54.550025 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:54.549366  3742 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:54.554388 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.555611  4082 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:54.599363 30985 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:54.599465  4059 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.624253 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:54.601656  4051 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.627384 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.579038 31061 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:54.605804 31144 rpcz_store.cc:276] 0709 12:25:53.314903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.315028 (+   125us) service_pool.cc:224] Handling call
0709 12:25:54.507384 (+1192356us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400063454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:54.507543 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679410048}
I20260709 12:25:54.539451 30988 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:54.548237 30983 raft_consensus.cc:2368] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:54.629499 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.623175 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:54.628180 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:54.632007 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:54.632572 31693 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:25:54.633533 31062 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:54.634089 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:25:54.633893 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 88242.7us
I20260709 12:25:54.635429 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:54.636059 30955 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:54.640220 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.640307 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.641299 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.642023 31058 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:54.643589 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.644166 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.646577 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.647061 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:54.648207 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:54.648401 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.648777 31696 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
W20260709 12:25:54.670037 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.677511 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.677749 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.589978  4025 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.539433  3964 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.678521 30991 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:54.678514 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.688505 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:54.608419  4044 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.680600 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.690454 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.690989 31143 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:54.691090 31144 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:25:54.692432 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.685173 31140 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:54.680657 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.680176 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.639871  4093 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:54.639984  3907 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.651811  3808 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:54.615163  4081 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:25:54.681804 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:54.643227  3996 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:54.710840  3591 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:54.711256  3591 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.711601  3591 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.712340 31143 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:54.677023  3891 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:54.682448  3984 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:54.682855  4083 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:54.682592  4059 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.721154  4059 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.694872  4082 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:54.727231 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:54.704614  3808 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.727902  3808 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.727994 30985 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:54.728873 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.693034 31137 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:54.651450  4068 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.651319  4037 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:54.701869 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:54.693872 30992 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:54.703337 31690 raft_consensus.cc:2368] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:25:54.708531 31061 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:25:54.708876 31698 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:25:54.737532 31060 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:54.729907 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.746884 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2417) took 1728 ms. Trace:
I20260709 12:25:54.736625 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.624392  3742 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.751922 31061 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:54.747216 30989 rpcz_store.cc:276] 0709 12:25:53.018476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.066085 (+ 47609us) service_pool.cc:224] Handling call
0709 12:25:54.746765 (+1680680us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540014e1d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:54.746845 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2574354304}
W20260709 12:25:54.749676 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60768us
W20260709 12:25:54.718037  4053 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.928s	user 0.001s	sys 0.001s
I20260709 12:25:54.751976 31062 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:54.688656  3814 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:54.759893  4089 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:25:54.759553 31690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:54.692181  3894 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:54.687346  4071 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:25:54.735308  3891 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.774309  3984 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:25:54.779089  4054 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:54.802117  4054 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:54.803124  4054 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.781237 31109 leader_election.cc:400] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:54.805768 31109 leader_election.cc:403] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:25:54.806762 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:54.807019 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2500) took 1046 ms. Trace:
I20260709 12:25:54.807159 30989 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:54.807389 31135 rpcz_store.cc:276] 0709 12:25:53.760479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.776160 (+ 15681us) service_pool.cc:224] Handling call
0709 12:25:54.508241 (+732081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.603459 (+ 95218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.806852 (+203393us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b5400063450. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:54.806966 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1682863872}
I20260709 12:25:54.807874  4068 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.788339  3814 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.790014 31137 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:54.793434  3891 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.816562 31064 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:54.817688  3891 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.796590  3742 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:54.776573 31690 raft_consensus.cc:606] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:54.819994 31690 raft_consensus.cc:614] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:54.796726 31065 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:25:54.822314 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:54.822885 31061 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.826620 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 1144) took 1031 ms. Trace:
I20260709 12:25:54.826988 31695 rpcz_store.cc:276] 0709 12:25:53.795360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.795533 (+   173us) service_pool.cc:224] Handling call
0709 12:25:54.826426 (+1030893us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54005c1a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:54.826545 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961714944}
I20260709 12:25:54.828687  4068 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:25:54.829641 31061 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:54.830377 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:54.833515 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.834084 31138 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.779958 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.828693 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2675) took 1644 ms. Trace:
I20260709 12:25:54.835296 30986 rpcz_store.cc:276] 0709 12:25:53.183860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.525906 (+342046us) service_pool.cc:224] Handling call
0709 12:25:53.648065 (+122159us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.606283 (+958218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.828452 (+222169us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b540029aed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:54.828634 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2073263488}
I20260709 12:25:54.804510  4054 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:54.653950  4051 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.844126 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.860463 31061 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:54.866061 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.866827 31140 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:54.867587 31661 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:54.867934 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:54.870937 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.874629 30987 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:54.875098 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:54.903514 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.904862 31061 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:54.907066 31140 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:54.908326 31141 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:54.810434  3893 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:54.914175 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:25:54.916123 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 229954us
I20260709 12:25:54.916774 31140 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:25:54.919016 31140 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:54.919112  3808 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.922428 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.923252  3808 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:54.924322 31135 raft_consensus.cc:2368] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15.
I20260709 12:25:54.926740 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:54.927501 30992 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.766722  4079 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 12:25:54.865559  3802 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.867812  3891 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:25:54.948704 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60b6187f32774daba89763099dabeb2a"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:54.949291 31695 raft_consensus.cc:606] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:54.949589 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1001) took 1877 ms. Trace:
I20260709 12:25:54.912559 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:54.934880 31138 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:25:54.932364  4037 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:54.950023 30990 rpcz_store.cc:276] 0709 12:25:53.072304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.072416 (+   112us) service_pool.cc:224] Handling call
0709 12:25:53.096960 (+ 24544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.133418 (+ 36458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.133455 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.181853 (+ 48398us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.531401 (+349548us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:53.792597 (+261196us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.800494 (+  7897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.800590 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.283666 (+483076us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b4800080058. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef07bdaa1 00007f2ef07b4b37 00007f2ef07b49ca 00007f2ef08ac26f 00007f2ef08abf37 00007f2ef08c055c 00007f2ef08c0508 00007f2ef08c0488 00007f2ef08c0443 00007f2ef08bf6ec 00007f2eef7e9a96
0709 12:25:54.717438 (+433772us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:25:54.949376 (+231938us) raft_consensus.cc:1685] finished
0709 12:25:54.949397 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.949509 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31144,"prepare.queue_time_us":446871,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":34386,"raft.run_cpu_time_us":29034,"raft.run_wall_time_us":1194697,"spinlock_wait_cycles":116710528,"thread_start_us":461623,"threads_started":2,"wal-append.queue_time_us":530046}
I20260709 12:25:54.955217 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.955819 31140 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
W20260709 12:25:54.956743 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.930262 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:54.876493  3742 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.960407 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 270918us
I20260709 12:25:54.964632 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.944595  3962 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:54.932895  3913 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:54.971894 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.979319 31138 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:54.958369 31106 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:54.974649 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:54.981233 31140 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:54.984130 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.984920 31140 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:54.973765 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:54.990605  4079 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:54.993093  4079 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:54.994168  4079 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.994664  4079 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.997085  4079 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:54.997702  4079 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:54.962118  4032 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:54.960209  4037 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.998807 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:54.910462  4083 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:25:54.944852  3802 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:54.953855  3638 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:25:54.963388  4064 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:25:55.004031  3962 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:54.987152  3913 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:54.974332  3742 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.005991  4037 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:55.018499  4018 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:25:55.030197 30992 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
W20260709 12:25:55.045972 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 87671.8us
I20260709 12:25:55.047134 31060 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.048396 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:54.970738  3901 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:25:55.024765 31138 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:54.972206  4093 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:54.966424  3925 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:25:54.998119  4079 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:55.027942 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1000) took 1956 ms. Trace:
I20260709 12:25:55.059276  4093 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.059394 30984 rpcz_store.cc:276] 0709 12:25:53.071408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.071528 (+   120us) service_pool.cc:224] Handling call
0709 12:25:53.095146 (+ 23618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.146556 (+ 51410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.146600 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.147233 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.720628 (+1573395us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b180037c180. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:25:54.918272 (+197644us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:54.918944 (+   672us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.932733 (+ 13789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.932839 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.027631 (+ 94792us) raft_consensus.cc:1685] finished
0709 12:25:55.027726 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.027871 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42874,"prepare.queue_time_us":284,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":46429,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":14904,"spinlock_wait_cycles":12353408,"thread_start_us":197457,"threads_started":1,"wal-append.queue_time_us":1855534}
I20260709 12:25:54.971632  3976 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:25:55.061621  3964 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.062041  3976 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:55.062470  3976 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:55.063323  3976 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.064577  3976 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:55.039510  3802 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.043009  4083 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:55.044395  4051 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.056304  3742 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.066527 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:55.066983 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
W20260709 12:25:55.057811  4097 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
I20260709 12:25:55.069895  4083 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:55.072407  3925 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:25:55.059396  4095 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:55.074913 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:55.075870 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:55.076396  4059 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.036510  4025 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:55.077732 31058 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:55.036216  3962 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.067445  3982 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:55.061625  3358 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:55.048193  4023 raft_consensus.cc:2764] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:25:55.082301 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 968) took 1942 ms. Trace:
I20260709 12:25:55.082932 31064 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:55.086717 31140 raft_consensus.cc:2368] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
W20260709 12:25:55.159924 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.088382 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 975) took 1097 ms. Trace:
I20260709 12:25:55.039120  3913 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:55.161903 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.160597 31141 rpcz_store.cc:276] 0709 12:25:53.991269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.116441 (+125172us) service_pool.cc:224] Handling call
0709 12:25:54.132194 (+ 15753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.908811 (+776617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.088164 (+179353us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b540001ac54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:55.088324 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1784396544}
I20260709 12:25:55.157970 31136 rpcz_store.cc:276] 0709 12:25:53.139588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.147672 (+  8084us) service_pool.cc:224] Handling call
0709 12:25:53.148060 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.814413 (+1666353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.814442 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:54.814481 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:54.814513 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.082078 (+267565us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400062f54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:55.082123 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.082246 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3831812864}
W20260709 12:25:55.168081 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 82432.2us
I20260709 12:25:55.168104 31060 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:55.175397 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:55.177033 30991 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:55.179131 31064 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:55.181316 31138 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:25:55.184970 31690 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:55.185360 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:55.187021  4066 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:55.195595 31694 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:55.197299  3591 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.200306 31142 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:25:55.115412  3925 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:55.123692  4021 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.159093  3358 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:55.207566 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:55.216665  4079 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:55.224529 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 240331us
I20260709 12:25:55.225262 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:55.227202 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:25:55.227607 31662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.001s	sys 0.018s
W20260709 12:25:55.228721 31662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.001s	sys 0.018s
I20260709 12:25:55.229705 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:55.232101 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:55.233165 30991 raft_consensus.cc:2368] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:55.223120  4079 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.231679  4038 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:55.234977 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:55.229147  3901 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:55.241758 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:55.236959 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.227424  4023 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:55.237843 31696 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:55.234920 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:55.203655  4037 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.242167 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:55.247352 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.238951  4038 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:55.224164  4021 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.234004  4027 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:25:55.243281  3853 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:55.241182  3901 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.259166  3591 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:55.259227  4068 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:55.245362  3925 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.263721 30988 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:25:55.244241  4062 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:55.251089  3638 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:55.250681  3913 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:55.254870  4025 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:55.272930 31689 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:25:55.274024  4007 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:55.226745  4083 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.264202  4038 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:55.276166 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:55.270845  3853 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:25:55.288372  4028 consensus_queue.cc:1175] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce27a72886964e4287059a1d66bd5b6b" responder_term: 16 status { last_received { term: 15 index: 14 } last_committed_idx: 14 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:55.289742  3853 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:55.291754  4094 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.308s	user 0.002s	sys 0.000s
I20260709 12:25:55.268476  3901 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:55.283339  4041 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.278671  3913 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:55.283736  3679 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:25:55.285204  4083 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:55.285609  3982 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:55.275182  4062 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.287334  4037 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.296046 31065 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:25:55.308799 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 61514.1us
I20260709 12:25:55.296571 31139 raft_consensus.cc:2410] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:55.300735 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:55.304776  3982 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.306273 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:55.302476  3913 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.315853 31137 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:55.302320  3932 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:55.251888  3358 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.318084 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.272624  4068 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.322003 31135 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:55.323586 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:55.325976 31696 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:55.326519 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:55.312160  4066 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.320649  3591 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:25:55.332914  4021 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.333483  4021 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.333937  4021 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.336062  4021 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.337999 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:55.316658  4023 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.316504  4037 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:55.328666  3984 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:55.350473  4068 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:55.352473  4066 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.395301  4066 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:55.365736  3982 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:55.367089  4062 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.323021  4006 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.397006  4006 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.323009  3932 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:55.375290 31068 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:55.379959  4038 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.384231  4064 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:55.339697  3907 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:25:55.385042  4070 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:55.386137  3679 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:55.383525 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:55.339529  3964 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:55.390406 31695 raft_consensus.cc:606] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:55.417529 31695 raft_consensus.cc:614] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:55.327895  3581 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:55.352769  3358 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.355966  4023 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.423456  4006 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.424075  4023 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.424249  3932 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.424800  3984 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:55.425495  3984 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:55.352909  4007 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:25:55.426644  3984 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.433516  4038 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.437389 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.439074 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:25:55.439376 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.440454  3581 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:55.440806  4068 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:55.441161  3581 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:55.441351 31059 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:55.441743  3581 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:55.441689 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:55.442461 31065 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:25:55.446602 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:55.457473 31060 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:55.451686 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:55.458591 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:55.459116 30987 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:25:55.459194 30985 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.459517 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:55.463006 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 79573.1us
I20260709 12:25:55.463750 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:55.464829 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:55.467326 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:55.469426 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:55.464346 30983 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:25:55.470481 31106 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:55.441892  4066 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:55.477387 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:55.480484 31067 raft_consensus.cc:2410] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:25:55.476271 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:55.481209 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:55.478582 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:55.490801 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:55.491153 31689 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:25:55.492746 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:55.494552 31060 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:25:55.478821  4059 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:55.479805  4066 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:55.505308 31059 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:55.462438  3901 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.442730  3581 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.480898  4070 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:25:55.526607  3913 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.505244  4059 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.530691  4059 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:55.530772 30986 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:55.532677 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:55.533367 30986 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:55.534695 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:55.535358 31061 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:25:55.524019  4076 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:55.534879 30992 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:55.479168  3591 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
W20260709 12:25:55.528889  4124 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:25:55.547180 30984 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:55.534083 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:55.544404  3679 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:55.553339 31059 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:55.557072  4106 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.328s	user 0.002s	sys 0.000s
W20260709 12:25:55.557811  4127 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:25:55.577183  4025 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:25:55.574641  4126 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:25:55.566699 30985 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:55.582890 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.584365  3901 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.584573 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:55.585146  4066 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.586244  4066 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.586438  4082 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:55.586915  4082 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:55.587306  4082 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.590548 31059 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:55.590456  4079 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:55.542922  4076 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:55.596793 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:55.598253 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:55.601063 31033 leader_election.cc:400] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:55.601439 31033 leader_election.cc:403] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.594255  4079 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.603261 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:25:55.602072  4129 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:25:55.609405 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:55.604766  4043 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:55.608765  4079 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:55.619069 31033 leader_election.cc:400] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:25:55.622212 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.624799 31033 leader_election.cc:403] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.612668  4093 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:55.612696  3742 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:55.635082 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.608566  3770 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:25:55.629931  4093 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:55.643352  4043 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:55.647380  3893 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:55.662506 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:55.660943 31142 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:25:55.649401  4099 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:55.659606  3742 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:55.666844  3742 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:55.667176  4109 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:25:55.676153  4043 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.659885  4093 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:55.676963  4105 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.663278  3913 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.660658  3770 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:25:55.651226  4044 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:25:55.685189  4068 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:55.689680 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.692746  3742 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.695724 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:55.715987 31691 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:25:55.703354 31138 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:55.702467 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:55.717798 31139 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:55.708392 31135 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:25:55.712085 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:55.719259 31144 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:55.711337  4068 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:55.715615  4020 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.724839  4099 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:55.724058  4079 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.731412  3742 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:55.728557  4082 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.747558  4059 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.750419  4068 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.751634  4068 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.754623 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:55.755159 30988 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:55.762496  4099 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:55.763025  4099 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.763796  4093 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:55.765232  4099 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.765753  4093 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.772308 30984 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:55.773383 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:55.773662 30987 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:55.774725 31137 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:55.774916 30985 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:25:55.775272 30985 pending_rounds.cc:85] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 11
I20260709 12:25:55.775456 30985 pending_rounds.cc:107] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:25:55.775597 31695 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:55.775713 30985 raft_consensus.cc:2889] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:55.776010 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:55.776649 31135 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:25:55.778384 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:55.779663 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.782519 31144 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:55.788497 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:55.789278 31064 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:55.785876 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:55.791477 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:55.793315  3581 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:55.800519 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:55.798255  4030 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:55.801736  4030 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:55.806485 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.799587 31664 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:55.812542 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:55.802124  4030 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:55.795622  4111 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:25:55.817623  4111 consensus_queue.cc:1243] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:55.818136  4111 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:55.796195  4099 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:55.826877 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 427209us
I20260709 12:25:55.827961 30958 leader_election.cc:400] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15.
I20260709 12:25:55.857492 30988 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:25:55.852006 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:55.819240  4025 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:55.819521  4066 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:55.822891  4057 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:55.822552  3638 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:25:55.864511  4066 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:55.927429 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:55.934754 30988 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:25:55.934767 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:55.935312 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:55.850708  4028 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:25:55.825815  4115 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:25:55.907989  4025 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:25:55.939610 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 78325.1us
I20260709 12:25:55.902379  4005 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:55.860790  4037 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:25:55.860211  3901 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
W20260709 12:25:55.929746  4117 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:55.904057  3893 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:55.846849  4076 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:55.936439  4066 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.918538  4057 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:55.944442 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:55.944622 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:55.972477  4143 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:55.969211  4059 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:55.948948  4128 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:55.983425 31142 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:55.989991  4082 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:55.990518  4079 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:55.991474  4151 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:25:56.002537 31068 raft_consensus.cc:2410] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:25:56.004593 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 61501.8us
I20260709 12:25:56.005486 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:55.986795  4054 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.949227  3932 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:25:55.988726  4105 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
I20260709 12:25:55.996692  4076 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:56.020337 31691 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:56.013324  4025 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.025292  4066 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.028123  4066 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:56.042011 31060 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:56.049887 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:56.051340 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:56.050763 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:56.057953 31137 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:56.049319 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:56.062196 31143 raft_consensus.cc:2410] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:25:56.063002 31144 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:56.068902 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:56.069434 31060 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:56.073926 30989 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:25:56.021929  4027 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:25:56.077301 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 70766.4us
I20260709 12:25:56.080117 30992 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:25:56.089542 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:56.013793  4054 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.059969  3808 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.100164 31060 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:25:56.101289 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:56.083808  4027 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.094342  4059 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.106843 31694 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 14: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:56.010636  4057 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.109319  4059 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.117214  4030 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.121196 31136 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 12:25:56.121614 31136 pending_rounds.cc:85] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 11
I20260709 12:25:56.121847 31136 pending_rounds.cc:107] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:25:56.122104 31136 raft_consensus.cc:2889] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:56.127758  4054 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.129225  4030 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.099946  4037 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:25:56.103299  3808 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.140280  3808 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:56.106084  4027 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:56.142529 31064 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:56.145044  3893 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:56.140493 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:56.148187  4038 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.148736  4038 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.150455  4027 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.151227  4038 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.158660  4027 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.169425 31033 leader_election.cc:400] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:25:56.170845 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:56.172483  3742 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:56.173772  3925 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:56.175408 31062 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:25:56.176302 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:56.176573 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:56.177124  4082 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:56.177532  4066 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:56.178085  4066 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:56.178344 31033 leader_election.cc:400] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14.
W20260709 12:25:56.179147 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 88958us
I20260709 12:25:56.178954  4066 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.179350  4082 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:56.180531  4043 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:56.180861  4043 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:56.181058 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:56.181697  4043 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.182199  4100 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.182991 31067 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:56.183255  4100 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.184228 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:56.185145 31067 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:56.179456 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:56.186592 31697 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:25:56.188762 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 142900us
I20260709 12:25:56.191671  3969 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:56.192011  4100 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:56.194684  4079 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:56.197752 31696 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:25:56.199127  4061 raft_consensus.cc:993] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:56.199638  4061 raft_consensus.cc:1081] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:56.200178 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:56.200800 31139 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:56.202811 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:56.202993 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:56.203341 31139 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:56.203716 31141 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:25:56.191805  4141 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:56.197882 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:56.216322  4165 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:56.216425 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1154) took 1759 ms. Trace:
I20260709 12:25:56.216792 31697 rpcz_store.cc:276] 0709 12:25:54.456943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.458802 (+  1859us) service_pool.cc:224] Handling call
0709 12:25:54.459097 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.186993 (+1727896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.216206 (+ 29213us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54006b0dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:56.216366 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3972049152}
I20260709 12:25:56.205711 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:56.181736 31063 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:25:56.218997 31696 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:56.208002 31142 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:25:56.219798 31139 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:56.208731 31143 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:25:56.222235 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.173669  3742 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:56.228751 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:56.229723 30990 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:56.215273  4141 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:56.223169 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:56.235321 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:56.231616 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:56.238257 31139 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:56.242148 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:56.245985 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:56.232669 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:56.233878 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:56.247661 31142 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:56.241982 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:56.252738 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:56.253783 31697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:56.252164  4099 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.249214  3770 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:56.252059  4159 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:56.241681  4005 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:56.266302 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:56.288322 31141 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:56.265054  3901 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:56.293602 31696 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:25:56.249702  4079 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:25:56.295687 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:56.271268  3770 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:56.264662  4099 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.288404 31067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60b6187f32774daba89763099dabeb2a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:25:56.300683 31067 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:56.301157 31067 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.303009 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:56.304935 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.306043 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:56.318491  4064 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:56.318888  4064 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.319240  4064 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:56.321303 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.328380  3996 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:56.330564 31062 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:56.330425 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:56.331599 30987 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:56.336488 30985 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:56.337563 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:56.339860 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:56.338905 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:56.341501 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:56.342175 31696 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:56.339656 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:56.303040  3770 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:56.296713  4163 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:56.287504  4005 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.353191 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:56.353962 31694 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:56.302291  4028 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:56.358522 31693 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 14: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:25:56.360191 31142 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:25:56.360477 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:56.364341 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:56.368623 31063 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:25:56.337249  4041 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:56.329794  4144 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:56.348948 31067 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.355798  3891 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:56.354441  4054 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:56.318903  3984 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:25:56.383306 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:56.359112  4028 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.289259  4141 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.355168  3808 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:56.362597  4027 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.366493  4051 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:56.394502 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:56.392870 31067 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:56.381877  4099 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:56.404331 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:56.405076 30990 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:56.407079 31698 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:25:56.407507  4172 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 12:25:56.408936  4168 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:25:56.400017 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 74566.8us
W20260709 12:25:56.409780  4167 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:25:56.355003  3893 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 12:25:56.410836 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 173541us
I20260709 12:25:56.371685  4041 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:56.417313 31143 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:25:56.413067 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:56.376415  4157 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:56.398865  3891 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:56.372095  3913 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:25:56.431246 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:56.398063  4028 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:56.426124 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50748.5us
I20260709 12:25:56.400342  3808 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.379308  4093 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:25:56.379963  3679 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:56.254346  4122 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:25:56.252022  4164 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:25:56.313410  4023 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:56.378391  4144 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.452360  4023 consensus_queue.cc:1243] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:56.453891  4144 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:56.454097  4169 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:56.454551  4144 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:56.454944  4144 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:56.455355  4144 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:56.455826  4159 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:56.456354  4159 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.412407  4027 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.434348  4079 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:25:56.434434  3808 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.459942  4027 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.460156  4043 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:56.461165 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:56.433113  4173 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:56.451654  4020 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:25:56.436808  4178 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:25:56.463474  4178 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:56.463792  4178 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:56.464538  4178 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:56.470903 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.416150  4051 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.438656  4157 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:56.466095  4178 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:56.494275 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 63132.1us
I20260709 12:25:56.491932  4159 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.497527  4144 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.482061  3925 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.523721 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 104699us
I20260709 12:25:56.531301 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:56.532174 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:56.537104 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:56.449926  3679 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.540287 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:56.541530 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:56.542281 31139 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:56.543813 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:56.546154 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:56.515283  4064 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.549464  4054 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:56.549866  4054 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.550205  4054 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.550392 30989 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
W20260709 12:25:56.551412  4185 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:25:56.552436  4051 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:56.558408 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 147777us
I20260709 12:25:56.470404  4041 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.545246  4173 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:56.574039 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:56.544935  4076 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:56.533819  3925 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.579150  4173 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:56.548125  4159 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:56.593184 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 61075.9us
I20260709 12:25:56.595230 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:56.595780 31694 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:56.541322  4082 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:25:56.610679 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:56.612062 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:56.596071 31694 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:56.614601 30992 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.615306 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:56.615787 31691 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:25:56.618120 30990 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:56.626308 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:56.560002  3932 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:25:56.645762 31141 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:25:56.583375  4051 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:56.613066  4041 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:56.608206  4076 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:56.613224  4066 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:56.653482 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:56.653971 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 94373.8us
I20260709 12:25:56.650682 31694 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.657788 31694 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.632611  4144 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.664638 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:56.665278 30988 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:25:56.539281  3891 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.580232  4173 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.666007 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:56.667824 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:56.668802 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.671232 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:56.680349 30992 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:56.680933 30990 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:56.656123  4064 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.683182 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:56.688831 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:56.582808  4157 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.656037  4128 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:25:56.751880  3679 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:56.654851  4051 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.674331  4144 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:56.762342  4076 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.687150  3808 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.684618  4197 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
W20260709 12:25:56.767737 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.667292  4193 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.757772  3770 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.763788  3925 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:56.791327  4125 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.796482  4184 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.815207 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 132121us
I20260709 12:25:56.818452 31059 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:56.812238  3925 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.824091  3982 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:56.960776 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.830942  4128 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.811357  4193 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:56.814915  4125 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:57.031394 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 361627us
I20260709 12:25:56.819521  4184 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:56.958727  4043 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.034330  4194 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:56.830847  3925 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.038762 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:57.042832  3358 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:57.043912  4057 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:57.029102  4082 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:57.054122 31694 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:25:57.060743  3808 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.061796  3770 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.063549  4184 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.063714  4125 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.065096  3925 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.058550  3358 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:57.076710 31694 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 12:25:57.077643 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:57.084036 31694 pending_rounds.cc:85] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 14
I20260709 12:25:57.084288 31694 pending_rounds.cc:107] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 12:25:57.084506 31694 raft_consensus.cc:2889] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:57.080861 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 408168us
I20260709 12:25:57.069679  4082 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.085984  4203 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:57.086526  3932 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:57.088783  4184 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.089267  4184 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.091300  4184 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:57.091635  4028 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.001s
I20260709 12:25:57.092039  4028 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.095319 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:57.095847 31062 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:57.096662 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:57.092351  4059 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.100179 31065 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:57.046768  4043 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.033813  4128 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:57.124403 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.105122  4023 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.105742  4189 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:57.125999  4028 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.140338  4082 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.141798  4128 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.121980  4051 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.126330  4195 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.167430 31062 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:25:57.169123 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:57.134130  4023 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.188560  4175 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:57.188971  4175 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:57.189777  4175 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.190774  4175 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:57.191359  4178 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.000s
I20260709 12:25:57.192433  4193 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.188575 31697 raft_consensus.cc:606] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:57.193936 31697 raft_consensus.cc:614] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:57.184247  4028 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:57.195211  4028 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:57.196086  4028 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.136310  4189 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:57.202013  4041 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.000s
I20260709 12:25:57.147719  3932 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:25:57.133351  3358 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:57.219161 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 95546.3us
I20260709 12:25:57.213035  4193 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:57.223033 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:57.227136 31143 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.230515 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:57.231935 31062 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:25:57.233347 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:57.153859  4195 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:57.235565 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.235785 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:57.216606  4051 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:57.237777 31142 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:57.244177 31143 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
W20260709 12:25:57.250936  4054 consensus_meta.cc:287] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.700s	user 0.009s	sys 0.001s
I20260709 12:25:57.251482  4054 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.253960  4054 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.258931  4005 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.252163 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 810) took 1043 ms. Trace:
I20260709 12:25:57.265178 31691 rpcz_store.cc:276] 0709 12:25:56.208756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.208888 (+   132us) service_pool.cc:224] Handling call
0709 12:25:56.615537 (+406649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.616170 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.251943 (+635773us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b540076c350. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:57.252103 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934565632}
I20260709 12:25:57.235759 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:57.271387 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:57.273275 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:57.274055 31059 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:25:57.276782 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:57.270339  4066 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.269039  3982 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:57.269348  4043 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.317808  4030 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:57.235021  4193 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.319643  4030 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.321669  4193 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.321833  4030 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.324103 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:57.325718 31059 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:57.326704 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:57.327248 31691 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:57.327685 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.275966  4005 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:57.324751 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.354910 30985 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:57.328265 31695 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:25:57.328747 31661 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:57.329802 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.269809  4051 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:57.451609 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.272873  3591 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:25:57.315390  4041 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:57.325337  4066 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:57.458137 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 128842us
W20260709 12:25:57.462146 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 133472us
I20260709 12:25:57.463140 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:25:57.463933 30992 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:57.326084  3891 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.470957 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:57.431951  4005 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:57.475915 31068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2570) took 1676 ms (client timeout 1758 ms). Trace:
W20260709 12:25:57.477190 31068 rpcz_store.cc:269] 0709 12:25:55.795382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.797179 (+  1797us) service_pool.cc:224] Handling call
0709 12:25:57.468659 (+1671480us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54002cd8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:57.471730 (+  3071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434592000}
I20260709 12:25:57.461740  4203 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:57.479013  3638 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:57.483342  3638 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.483304  4203 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:57.485176 31017 heartbeater.cc:646] Failed to heartbeat to 127.29.37.126:44053 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.29.37.126:44053 timed out after 15.000s (SENT)
I20260709 12:25:57.486313  4178 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:57.487167  4115 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:57.487687  4115 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.489799  4115 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.469590  4043 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.485937  3638 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.465235  4041 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.327095  3982 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.478866  4025 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:57.468677  4057 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:57.502055  4041 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:57.508021 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.481287  4005 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:57.503710  3982 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.504119  4025 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:57.505638 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:57.506027 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:57.517282 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:57.503502  4051 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.509057 30992 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:57.522631  4025 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:57.522956  4197 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:57.523411  4197 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.523734  4197 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:57.479115  4194 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.526839  4194 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.528980  4212 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:57.529454  4194 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.529484  4212 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.532181  4212 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.533475 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1168) took 1604 ms. Trace:
I20260709 12:25:57.533655  3901 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:57.534207  4082 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:57.534181  3901 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.534849  4082 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:57.535758  4082 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.536747 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2378) took 1174 ms. Trace:
I20260709 12:25:57.537081 31136 rpcz_store.cc:276] 0709 12:25:56.362373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.362496 (+   123us) service_pool.cc:224] Handling call
0709 12:25:56.376171 (+ 13675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.480621 (+104450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.480657 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:56.480698 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:56.480736 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.536535 (+1055799us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400063e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:57.536581 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.536695 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239729536}
I20260709 12:25:57.536376  3901 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.538758 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:57.539108 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:57.539897 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.540309 31144 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:25:57.540452 30989 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:57.541419 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.541673 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.541894 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:57.533859 31698 rpcz_store.cc:276] 0709 12:25:55.928913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.929090 (+   177us) service_pool.cc:224] Handling call
0709 12:25:55.940157 (+ 11067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.407436 (+467279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.533237 (+1125801us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54004a24d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:57.533423 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073143040}
I20260709 12:25:57.551949 31060 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:57.557487 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.558370 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:57.558959 31136 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:25:57.476810  3891 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.565127 31698 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:25:57.567842 31691 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:57.571569 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:57.505884  4043 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.555177  4038 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:57.553251  3638 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:57.515626  4057 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.558600  4051 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:25:57.580693 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.542508  4206 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:25:57.575503  4079 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:57.589118  4082 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
W20260709 12:25:57.597667  4182 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.835s	user 0.006s	sys 0.000s
I20260709 12:25:57.579528  4194 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:57.576874  3982 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:57.573890  4093 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:25:57.584137  3853 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:57.604843 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2446) took 1200 ms. Trace:
I20260709 12:25:57.609146  4105 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:57.609587  4105 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:57.610829  4066 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.611653  3891 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:57.612684  4215 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:25:57.612708 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.613004 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59170 (request call id 12) took 1378 ms. Trace:
I20260709 12:25:57.613300  3853 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.584409  4051 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.615525  3853 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.616977 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.617110  4051 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.617571 31698 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:25:57.605425  4194 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.605412 30983 rpcz_store.cc:276] 0709 12:25:56.404293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.404671 (+   378us) service_pool.cc:224] Handling call
0709 12:25:56.404987 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:56.405370 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.405397 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:56.405954 (+   557us) log.cc:844] Serialized 52 byte log entry
0709 12:25:56.556732 (+150778us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:56.556771 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.556834 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.597311 (+1040477us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b2400175fe8. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee862d9c4 00007f2ee862bacb 00007f2ee87e6dc9 00007f2ee87e6a55 00007f2ee87e609d 00007f2eeb41810e 00007f2eeb42f88c 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221
0709 12:25:57.603839 (+  6528us) raft_consensus.cc:1685] finished
0709 12:25:57.603882 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.604763 (+   881us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4493,"raft.queue_time_us":4870,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":4857728,"thread_start_us":150367,"threads_started":1,"wal-append.queue_time_us":250741}
I20260709 12:25:57.613443 31697 rpcz_store.cc:276] 0709 12:25:56.244355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.253633 (+  9278us) service_pool.cc:224] Handling call
0709 12:25:57.612805 (+1359172us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54004d4ed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88e853 00007f2ef085c60a 00007f2ee9de27d4 00007f2ee9de2762 00007f2ee9de2691 00007f2ee9de260c 00007f2ee9de18c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:57.612913 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2143849344}
I20260709 12:25:57.614890  3742 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:57.581539  3638 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.627628  3638 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.628456  4194 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.630569  4164 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:57.631157  4164 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.632169 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.624583  3742 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:25:57.632849 31690 raft_consensus.cc:2368] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:57.633323  4164 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.634524 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.635097 31692 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:25:57.638382  4189 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:57.638564 30955 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.66:45157 timed out after 1.758s (SENT) [suppressed 6 similar messages]
I20260709 12:25:57.645474  4043 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.624217  4125 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.646591 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:57.647172  4125 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.648095  4068 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:57.648602  4068 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:57.649121  4125 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.649466  4068 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.655858 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.658332 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:57.656206 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.669811 30990 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
W20260709 12:25:57.669104 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 94499.7us
I20260709 12:25:57.671901  4189 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:57.672422 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.672390  4189 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.673096 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.675163 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.675434 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:57.675767 30990 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:57.621531  3982 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.677670 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:57.678248 31067 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:25:57.678642  3982 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.680414  4105 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.681699  4208 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:57.681829  4105 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:57.682317  4208 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.684541  4208 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.685925  4041 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.687198 30988 raft_consensus.cc:2368] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20.
I20260709 12:25:57.688865 30992 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:57.669258 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:57.693922 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.601897  4195 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:57.696949 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:57.698642 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.700115  4194 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:57.700369 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.651130  4188 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:57.733847 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:57.689745  4164 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:57.702962 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.745316 30958 leader_election.cc:400] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:57.745648 30958 leader_election.cc:403] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:57.720389  4194 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.721004 31698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:25:57.721586  4057 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:57.633423  3742 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:57.668092  4025 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.667901  4043 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:57.692386  4051 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:57.727864  3891 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:57.732016 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.766860 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.747877 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:57.756044 31698 raft_consensus.cc:606] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:57.752341 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:57.762460 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:25:57.763302 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 189506us
I20260709 12:25:57.701017  3982 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:57.765481 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.802170 30992 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:25:57.802116 30984 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:25:57.803263 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.758677  4164 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.807591  4164 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:57.808658 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 56388.6us
I20260709 12:25:57.752274  4027 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:57.754904  4144 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.810407  4027 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.810817  4027 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:57.812813  4099 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:57.812935 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.801936  3891 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.813449  4099 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:57.732905  4188 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:25:57.817775 31691 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:25:57.814163  3891 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.761711  4176 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:57.820127 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:57.822153 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.822832 30989 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:25:57.823472 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:57.817795  3742 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:57.802379  4051 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.751634  4195 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:25:57.801944  3982 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.912787 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:25:57.958863 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.801097  4189 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.983002 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:25:57.992707 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 80020.1us
I20260709 12:25:57.993357 31142 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:25:57.993803 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:57.985193 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:25:57.988309 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 349747us
I20260709 12:25:57.817948  4164 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:57.996904 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:58.002429 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:58.005802 30984 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:58.004732 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:25:58.009894 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:58.011302 30992 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:25:57.804451  4043 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.015560 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:58.015787 30987 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:25:57.982077  4051 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:58.018049  4188 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.019124  4188 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:58.019861  4194 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:57.985842  4228 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:57.995394  3982 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.818531  4144 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.004490  4173 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:58.033121 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:57.997913  4193 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:58.004370  4041 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:58.004285  4164 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:58.004109  3591 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:58.008011  4178 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:58.036298 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:25:58.045540 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:58.015256  4066 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.053675 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:58.055215  4173 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:58.055436 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.055766  4173 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:58.056288  4173 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:58.057267 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.058315 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.059072 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.060087 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.060901 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.061460 31062 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:25:58.062454 31059 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:58.065521 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.066535 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.067601 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:58.067559  4173 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.068548 31062 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:25:58.029623  4189 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.084098 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 95799us
I20260709 12:25:58.046530  4164 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:58.048152  4178 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.071044  4093 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:58.095618  4177 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.096515  4177 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.098538  4177 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:58.090431 31106 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:25:58.100567  4177 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:58.103120 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:25:58.102188  4177 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:58.114864 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.116541 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:58.077821  4066 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.070655  3591 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.085583  4189 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.040792  4005 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.033142  4228 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.042855  4193 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.118170 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:58.137187 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:58.169540 30987 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:25:58.149590 31058 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:58.155226 30991 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:25:58.159101 31064 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
W20260709 12:25:58.160794 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57926.5us
W20260709 12:25:58.187594 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.163769 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 812) took 1563 ms. Trace:
I20260709 12:25:58.177693 31661 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:58.194151 31696 rpcz_store.cc:276] 0709 12:25:56.600844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.600990 (+   146us) service_pool.cc:224] Handling call
0709 12:25:56.601251 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.655498 (+ 54247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.655541 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:56.655581 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:56.655624 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.163572 (+1507948us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b54006ee6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:58.163612 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.163720 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124318080}
W20260709 12:25:58.145153  4229 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:25:58.190143  4197 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.001s
I20260709 12:25:58.133770  4093 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.143999  4043 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:58.127561  4144 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:58.160681  4177 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.142871  4099 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.139343  4176 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:25:58.152217  4194 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:58.130566  4025 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:58.125175  3591 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.150553  4193 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.094522  4178 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:58.159564  4005 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:58.122527  4066 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.161304  4173 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.132817  4189 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.153048  4204 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:58.148555  4228 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:58.216884  4125 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:58.227560  4068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.000s	sys 0.013s
I20260709 12:25:58.154270  4027 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.227576  4175 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:58.229763  4093 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:58.240626 31696 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:58.242740 30992 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:58.243721 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:58.226327  4240 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:58.246253 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:58.230038  4197 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:58.250504 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 63770.3us
I20260709 12:25:58.237823  4177 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:58.282203 30983 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:25:58.389322 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:58.394058 30987 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:25:58.234071  4242 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:58.396047  4242 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:25:58.397086 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1200) took 2057 ms. Trace:
I20260709 12:25:58.397151  4125 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:25:58.397614 31693 rpcz_store.cc:276] 0709 12:25:56.339517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.339633 (+   116us) service_pool.cc:224] Handling call
0709 12:25:58.396873 (+2057240us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5400542754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:58.397000 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39769856}
I20260709 12:25:58.241322  4099 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.157s
I20260709 12:25:58.398237  4125 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.398952  4099 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:58.399303  4099 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:25:58.386708  4051 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.401597 31139 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:25:58.341375  3982 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.390436  4226 raft_consensus.cc:993] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:58.403461  3969 raft_consensus.cc:1081] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:25:58.404356  4023 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:58.404904  3982 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:58.340293  4105 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.657s	user 0.007s	sys 0.002s
I20260709 12:25:58.405656 31067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:25:58.235447  4164 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.407925 31063 raft_consensus.cc:2368] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:25:58.409485  4197 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:58.395282  4068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.004s	sys 0.016s
I20260709 12:25:58.244740  4194 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.400077  4099 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.411839 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:58.412395  4099 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:58.412487  4194 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.329002  3891 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.435166 31144 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:25:58.436884  3891 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:58.413154 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:58.437427 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:25:58.438007  3891 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:58.438835  3891 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:58.441982  3893 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:58.414261  4251 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:58.402796  4051 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.414310 31067 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:58.442668  3893 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:58.281432  4224 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.673s	user 0.009s	sys 0.000s
I20260709 12:25:58.415742 30984 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:25:58.416553 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:58.444519 31059 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:25:58.444635  3893 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.448290  4068 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:58.448796  4068 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.450863  4068 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:58.451304  4068 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:58.451902 31059 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:25:58.451964  4068 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.453369 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:25:58.416739 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:58.396507  4242 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:59.924469 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2461) took 1508 ms. Trace:
I20260709 12:25:59.924844 30989 rpcz_store.cc:276] 0709 12:25:58.416380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.416500 (+   120us) service_pool.cc:224] Handling call
0709 12:25:59.924405 (+1507905us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.424985 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:59.932058 30986 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:25:59.932590 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2728) took 1508 ms. Trace:
I20260709 12:25:59.932897 30986 rpcz_store.cc:276] 0709 12:25:58.424595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.424744 (+   149us) service_pool.cc:224] Handling call
0709 12:25:59.932527 (+1507783us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.420320 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:59.934011  4079 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:59.934566 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2726) took 1514 ms. Trace:
I20260709 12:25:59.934966 30992 rpcz_store.cc:276] 0709 12:25:58.419929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.420085 (+   156us) service_pool.cc:224] Handling call
0709 12:25:59.934488 (+1514403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.423252 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:25:58.514653 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2715) took 1185 ms. Trace:
I20260709 12:25:58.478091 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:59.934592  4079 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:25:59.937456 31059 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2868) took 1471 ms (client timeout 1794 ms). Trace:
I20260709 12:25:58.463709 31062 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:00.040297 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2864) took 1577 ms. Trace:
I20260709 12:26:00.040683 31062 rpcz_store.cc:276] 0709 12:25:58.463075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.463221 (+   146us) service_pool.cc:224] Handling call
0709 12:25:58.463537 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.040037 (+1576500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.040238 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.464743 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:25:58.253806 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.467584 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.052201 31065 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
W20260709 12:26:00.052661 31065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2866) took 1588 ms (client timeout 1673 ms). Trace:
W20260709 12:26:00.052973 31065 rpcz_store.cc:269] 0709 12:25:58.465323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.467336 (+  2013us) service_pool.cc:224] Handling call
0709 12:26:00.052593 (+1585257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.523757 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.055948 31058 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:00.056329 31058 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2601) took 1548 ms (client timeout 1765 ms). Trace:
W20260709 12:26:00.056720 31058 rpcz_store.cc:269] 0709 12:25:58.507895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.523437 (+ 15542us) service_pool.cc:224] Handling call
0709 12:26:00.056272 (+1532835us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.057700 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:25:58.443073 31067 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:58.474010 30888 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 200) took 34505 ms (client timeout 15000 ms). Trace:
I20260709 12:25:58.426934 31136 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:00.061532 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2546) took 1635 ms. Trace:
I20260709 12:26:00.061864 31136 rpcz_store.cc:276] 0709 12:25:58.426024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.426338 (+   314us) service_pool.cc:224] Handling call
0709 12:25:58.426637 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.061295 (+1634658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.061481 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.427276 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.063580 31138 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
W20260709 12:26:00.064042 31138 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2549) took 1637 ms (client timeout 2023 ms). Trace:
W20260709 12:26:00.064355 31138 rpcz_store.cc:269] 0709 12:25:58.426946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.427063 (+   117us) service_pool.cc:224] Handling call
0709 12:26:00.063989 (+1636926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.428279 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:25:58.426981 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:00.066771 31143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2548) took 1640 ms (client timeout 1636 ms). Trace:
W20260709 12:26:00.067149 31143 rpcz_store.cc:269] 0709 12:25:58.426656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.426771 (+   115us) service_pool.cc:224] Handling call
0709 12:26:00.066703 (+1639932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.476967 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:00.069643 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 1.59271e+06us
I20260709 12:26:00.070293 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:25:58.488031 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.071543  4208 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:00.071749 31692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1215) took 1598 ms (client timeout 1673 ms). Trace:
I20260709 12:25:58.472432 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.072124  4208 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.471832 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.073707 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1211) took 1602 ms. Trace:
I20260709 12:26:00.073804  4270 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.074054 31697 rpcz_store.cc:276] 0709 12:25:58.471460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.471608 (+   148us) service_pool.cc:224] Handling call
0709 12:26:00.073649 (+1602041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.074357  4270 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.074992  4208 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.075177 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:00.075836 31697 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1214) took 1602 ms (client timeout 2121 ms). Trace:
I20260709 12:26:00.076404  4270 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:00.076252 31697 rpcz_store.cc:269] 0709 12:25:58.473233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.074948 (+1601715us) service_pool.cc:224] Handling call
0709 12:26:00.075786 (+   838us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.284351 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:00.071065 31067 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:00.079339 31698 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 820) took 1796 ms (client timeout 1715 ms). Trace:
I20260709 12:26:00.079746 31061 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:00.079698 31698 rpcz_store.cc:269] 0709 12:25:58.283451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.284042 (+   591us) service_pool.cc:224] Handling call
0709 12:26:00.079272 (+1795230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.473233 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:58.489853 31696 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:26:00.060225 30888 rpcz_store.cc:269] 0709 12:25:23.969208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.995809 (+ 26601us) service_pool.cc:224] Handling call
0709 12:25:58.473940 (+34478131us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:52.775075 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:52.775909 (+   834us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:52.775988 (+    79us) write_op.cc:435] Acquired schema lock
0709 12:25:52.776018 (+    30us) tablet.cc:662] Decoding operations
0709 12:25:52.777242 (+  1224us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:52.777374 (+   132us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:52.777416 (+    42us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:25:52.880633 (+103217us) tablet.cc:701] Row locks acquired
0709 12:25:52.919105 (+ 38472us) write_op.cc:260] PREPARE: finished
0709 12:25:52.951638 (+ 32533us) write_op.cc:270] Start()
0709 12:25:52.951876 (+   238us) write_op.cc:276] Timestamp: P: 1783599952951526 usec, L: 0
0709 12:25:52.951930 (+    54us) op_driver.cc:348] REPLICATION: starting
0709 12:25:53.174117 (+222187us) log.cc:844] Serialized 3194 byte log entry
0709 12:25:54.325369 (+1151252us) op_driver.cc:464] REPLICATION: finished
0709 12:25:54.436709 (+111340us) write_op.cc:301] APPLY: starting
0709 12:25:54.436825 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:54.572782 (+135957us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:54.572887 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:54.580386 (+  7499us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:54.580683 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 12:25:54.580726 (+    43us) write_op.cc:312] APPLY: finished
0709 12:25:54.581052 (+   326us) log.cc:844] Serialized 81 byte log entry
0709 12:25:54.581240 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:54.581354 (+   114us) write_op.cc:454] Released schema lock
0709 12:25:54.581408 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:54.607307 (+ 25899us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:54.619442 (+ 12135us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":824306,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":108383,"apply.run_cpu_time_us":11765,"apply.run_wall_time_us":147760,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37583,"num_ops":8,"prepare.queue_time_us":96122,"prepare.run_cpu_time_us":56685,"prepare.run_wall_time_us":896404,"raft.queue_time_us":135763,"raft.run_cpu_time_us":10596,"raft.run_wall_time_us":77901,"replication_time_us":1373274,"thread_start_us":576017,"threads_started":4,"wal-append.queue_time_us":642738,"wal-append.run_cpu_time_us":36164,"wal-append.run_wall_time_us":1704014}]]}
I20260709 12:25:58.442065  3891 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:00.086812  4270 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:00.087425  4270 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.455085  3893 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.474632  3925 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:58.428793 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:25:58.417356  4027 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:25:58.454377  4032 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:00.096154  4032 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:00.088536 31067 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.097110 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:38956 (request call id 1459) took 1691 ms. Trace:
I20260709 12:26:00.097409 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 1217) took 1604 ms. Trace:
I20260709 12:26:00.097071  4032 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:58.253364  4043 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:25:58.454116  4068 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.099030  4043 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.470148  4093 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.470243  4100 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:00.101950  4100 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.097640 31067 rpcz_store.cc:276] 0709 12:25:58.405417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.405535 (+   118us) service_pool.cc:224] Handling call
0709 12:26:00.087450 (+1681915us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b54001a6dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88ba3d 00007f2ef085bcd0 00007f2ee9de0254 00007f2ee9de01e2 00007f2ee9de0111 00007f2ee9de008c 00007f2ee9ddf342 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.097030 (+  9580us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18390400}
I20260709 12:26:00.104080  4100 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.104775 31060 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:25:58.415243  4141 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:58.416471  4164 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.393796  4177 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.118847  4177 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:58.354075  4178 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:58.418118  4204 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.121351  4178 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:25:58.431402  4222 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.827s	user 0.002s	sys 0.006s
I20260709 12:25:58.433029 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:25:59.213829  4245 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:00.125510 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 1.69244e+06us
W20260709 12:25:59.578805 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08003524e0 after lost signal to thread 29845
W20260709 12:26:00.126130 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08003697e0 after lost signal to thread 29848
I20260709 12:26:00.126257 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.127240 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.127810  4068 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:00.128005  3581 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:00.128392  3581 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:25:59.657800 30983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 1047) took 1895 ms (client timeout 2063 ms). Trace:
I20260709 12:25:59.663497  4051 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.128804  3581 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:00.128947 30983 rpcz_store.cc:269] 0709 12:25:57.762020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.762163 (+   143us) service_pool.cc:224] Handling call
0709 12:25:59.657536 (+1895373us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54003458d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:59.657716 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017774848}
I20260709 12:26:00.129179  4051 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.131400  4051 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:25:59.665455 30958 connection.cc:256] RPC call timeout handler was delayed by -1.1986871509998309s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
I20260709 12:25:59.685505 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:00.132732 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:59.691883  4228 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.134137  4051 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:00.134485  4228 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.134672  4051 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.136504  4228 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:59.698374  4066 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.137718 31068 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:26:00.137458  4066 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.138123 31068 pending_rounds.cc:85] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 12
I20260709 12:26:00.138419 31068 pending_rounds.cc:107] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:26:00.138659 31068 raft_consensus.cc:2889] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.140053  4066 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:59.730679  4163 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:59.865213  4258 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:26:00.140653 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1443) took 2596 ms. Trace:
I20260709 12:26:00.140923  4163 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.141089 31068 rpcz_store.cc:276] 0709 12:25:57.544731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.546808 (+  2077us) service_pool.cc:224] Handling call
0709 12:25:57.549876 (+  3068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.139089 (+2589213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.140394 (+  1305us) spinlock_profiling.cc:243] Waited 2.59 s on lock 0x7b5400384854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:00.140576 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5950673280}
I20260709 12:26:00.141708 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1190) took 2060 ms. Trace:
I20260709 12:26:00.142253  4203 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.143142  4163 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:59.889678 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58099.3us
I20260709 12:26:00.142057 31690 rpcz_store.cc:276] 0709 12:25:58.081843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.082064 (+   221us) service_pool.cc:224] Handling call
0709 12:25:58.089010 (+  6946us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.092122 (+  3112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.092158 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.731282 (+1639124us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b1800172b40. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87752a6 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a
0709 12:25:59.731674 (+   392us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.733738 (+  2064us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:59.776195 (+ 42457us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.776394 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.776463 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.141428 (+364965us) raft_consensus.cc:1685] finished
0709 12:26:00.141531 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.141655 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":329,"prepare.queue_time_us":67335,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":119292,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":13248,"spinlock_wait_cycles":3488384,"thread_start_us":41414,"threads_started":2,"wal-append.queue_time_us":913}
I20260709 12:26:00.144078 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.144649 31068 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
W20260709 12:26:00.145063 31068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2609) took 1635 ms (client timeout 2179 ms). Trace:
W20260709 12:26:00.145382 31068 rpcz_store.cc:269] 0709 12:25:58.509825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.143831 (+1634006us) service_pool.cc:224] Handling call
0709 12:26:00.145016 (+  1185us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:59.890002 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2459) took 1646 ms (client timeout 1926 ms). Trace:
W20260709 12:26:00.146965 30984 rpcz_store.cc:269] 0709 12:25:58.243282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.243484 (+   202us) service_pool.cc:224] Handling call
0709 12:25:59.889926 (+1646442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:59.893491  4037 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.147936  4037 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.148873  4163 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.895349  4122 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.150501  4122 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.153337  4122 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.153043  4163 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:25:59.904188  4236 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.799s	user 0.004s	sys 0.008s
W20260709 12:25:59.914498 30955 leader_election.cc:336] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Timed out: RequestConsensusVote RPC to 127.29.37.66:45157 timed out after 1.758s (SENT)
I20260709 12:26:00.154670 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:00.155268  4163 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:00.155486 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 241027us
I20260709 12:26:00.155786  4163 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:00.156653  4198 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:00.157153  4198 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:59.930367  4173 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.158536  4173 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.159425  4198 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.159919  4198 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.160543  4173 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:59.930492  4261 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:25:59.936568 30987 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2727) took 1513 ms (client timeout 1558 ms). Trace:
I20260709 12:25:59.936354 30988 rpcz_store.cc:276] 0709 12:25:57.329425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.329550 (+   125us) service_pool.cc:224] Handling call
0709 12:25:58.514429 (+1184879us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54000226d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:25:58.514586 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748540672}
W20260709 12:25:59.937831 31059 rpcz_store.cc:269] 0709 12:25:58.465934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.477838 (+ 11904us) service_pool.cc:224] Handling call
0709 12:25:59.937401 (+1459563us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.161201  4261 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.162104 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:00.162581 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.162886 31059 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.163303  4163 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:25:59.939106  4079 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:00.161221 30987 rpcz_store.cc:269] 0709 12:25:58.422857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.423008 (+   151us) service_pool.cc:224] Handling call
0709 12:25:59.936513 (+1513505us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:59.930884  4184 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.164124  4163 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.165544  4184 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.166981  4163 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.167505  4184 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:59.931363  4076 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.168179  4076 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.168887 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:59.939275  4205 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.170526  4205 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.172585  4205 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:59.941109  4242 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:59.953294  4125 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.173317  4125 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:59.952520 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2534) took 2726 ms. Trace:
I20260709 12:25:59.953940  4105 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:59.953940  4194 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.180729  4194 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.170151  4076 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:25:59.957669  4206 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:59.960352 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:25:59.966306  4197 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:25:59.969805  4225 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.163467  4261 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:00.186277 31694 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1216) took 1712 ms (client timeout 2087 ms). Trace:
I20260709 12:26:00.186504  4225 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:00.186731 31694 rpcz_store.cc:269] 0709 12:25:58.473841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.960120 (+1486279us) service_pool.cc:224] Handling call
0709 12:26:00.186201 (+226081us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.185622  4105 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.187705 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.188249 31694 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:25:59.977267 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.188714  4105 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:00.188879 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.153872  4122 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:59.984031  4264 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.196461  4163 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.196813  3913 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.984980  3358 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:00.197355  3913 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:25:59.991727  3853 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.198495  3853 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.196688 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.005008  3591 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.005587  4263 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.006209  4062 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:00.006738 31108 connection.cc:256] RPC call timeout handler was delayed by -2.4841970350003066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:26:00.012105  4193 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:00.027922 31063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2853) took 1975 ms (client timeout 1782 ms). Trace:
I20260709 12:26:00.021780 30882 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 13 to 18, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 18 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:00.032109 31691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1208) took 2419 ms (client timeout 1962 ms). Trace:
W20260709 12:26:00.220865 31691 rpcz_store.cc:269] 0709 12:25:57.612299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.612472 (+   173us) service_pool.cc:224] Handling call
0709 12:26:00.031914 (+2419442us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54006a09d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.032014 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469717760}
I20260709 12:26:00.221836 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.221485 30882 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 15, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 15 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:00.222434 31691 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 8.
I20260709 12:26:00.223309 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.223516 30882 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:00.224311 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:00.224861 31691 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:00.225194 31691 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:00.225066 30882 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 17, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 17 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:00.042297  4082 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.226539 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.046988 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:00.227327 31136 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:26:00.227384 31062 raft_consensus.cc:2368] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16.
W20260709 12:26:00.227902 31062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2612) took 1690 ms (client timeout 1758 ms). Trace:
W20260709 12:26:00.214133 31063 rpcz_store.cc:269] 0709 12:25:58.052190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.055203 (+  3013us) service_pool.cc:224] Handling call
0709 12:26:00.027640 (+1972437us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540027c254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.027831 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808881408}
W20260709 12:26:00.228245 31062 rpcz_store.cc:269] 0709 12:25:58.537537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.046757 (+1509220us) service_pool.cc:224] Handling call
0709 12:26:00.227849 (+181092us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.047478 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2541) took 2236 ms. Trace:
W20260709 12:26:00.048060 31064 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2865) took 1583 ms (client timeout 1900 ms). Trace:
I20260709 12:26:00.065565 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.071507  4265 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.081072 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1458) took 1689 ms. Trace:
I20260709 12:26:00.229290 31139 rpcz_store.cc:276] 0709 12:25:57.811068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.811431 (+   363us) service_pool.cc:224] Handling call
0709 12:25:57.811727 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.401962 (+590235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.047227 (+1645265us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b5400062a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:00.047422 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1356028288}
I20260709 12:26:00.081140  4115 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:00.235392  4115 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.235785  4115 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:58.417989  3808 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:00.072134 31692 rpcz_store.cc:269] 0709 12:25:58.473546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.487729 (+ 14183us) service_pool.cc:224] Handling call
0709 12:26:00.071688 (+1583959us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.239945  3808 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:00.083972  4054 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.240717  3913 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.241500  4054 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.241916  4054 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:00.084471 31695 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1213) took 1612 ms (client timeout 1558 ms). Trace:
I20260709 12:26:00.089382  4270 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.091122  3893 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.094249 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:00.246393 31695 rpcz_store.cc:269] 0709 12:25:58.472059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.473017 (+   958us) service_pool.cc:224] Handling call
0709 12:26:00.084389 (+1611372us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:00.247207 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 153094us
I20260709 12:26:00.101161  4043 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.231289  4173 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:00.107040 31060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2860) took 2053 ms (client timeout 2579 ms). Trace:
I20260709 12:26:00.107313  4093 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.247948 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:00.122273 31058 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.248323  4122 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:00.248008  4173 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.249449 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.125250  4245 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:25:59.512904  4249 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.226552  4082 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.250751  4173 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:00.247999 31060 rpcz_store.cc:269] 0709 12:25:58.054061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.059854 (+  5793us) service_pool.cc:224] Handling call
0709 12:26:00.106841 (+2046987us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54002cd8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.106977 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4700912896}
I20260709 12:26:00.251492  4245 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.097745 31697 rpcz_store.cc:276] 0709 12:25:58.474081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.077250 (+1603169us) service_pool.cc:224] Handling call
0709 12:26:00.077581 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.077815 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.077852 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.077909 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:00.077973 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.078043 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.078159 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.121999  4204 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.251941 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.128554  4068 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:59.662889  4099 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.252573 31060 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:00.252818  4099 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.253574 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.252563  4068 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.254969  4099 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.255069  4068 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.142841  4203 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.146570 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.256120 31068 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
W20260709 12:26:00.256605 31068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2870) took 1790 ms (client timeout 2237 ms). Trace:
W20260709 12:26:00.256954 31068 rpcz_store.cc:269] 0709 12:25:58.466542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.146349 (+1679807us) service_pool.cc:224] Handling call
0709 12:26:00.256549 (+110200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.257853 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.262969 31068 raft_consensus.cc:2368] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:26:00.263885 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:25:59.938910  4175 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.264744  4082 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.265162  4175 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.267163  4175 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:25:59.953554 29850 debug-util.cc:398] Leaking SignalData structure 0x7b080014c340 after lost signal to thread 31109
I20260709 12:25:59.953025  4262 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.263036 31058 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2614) took 1720 ms (client timeout 2068 ms). Trace:
I20260709 12:26:00.268968  4262 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:00.269600 31058 rpcz_store.cc:269] 0709 12:25:58.542176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.057399 (+1515223us) service_pool.cc:224] Handling call
0709 12:26:00.262870 (+205471us) spinlock_profiling.cc:243] Waited 63.8 ms on lock 0x7b54001bba54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.262987 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146688256}
I20260709 12:26:00.271361  4262 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:26:00.271976 31690 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:00.174953 31693 raft_consensus.cc:2410] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:00.174743 31141 rpcz_store.cc:276] 0709 12:25:57.225904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.228482 (+  2578us) service_pool.cc:224] Handling call
0709 12:25:57.242161 (+ 13679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.475340 (+233179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.475378 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.475431 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:57.475468 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.952292 (+2476824us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540029ead4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:25:59.952338 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.952436 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535732480}
I20260709 12:26:00.274441 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.275033 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.275537 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.275964 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.276132 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.276546 31068 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:00.276763 31058 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 14: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.277374 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.277559 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1212) took 1805 ms. Trace:
I20260709 12:26:00.205132  4076 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:26:00.149950  4037 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.182865  4194 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:25:59.979380  3770 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.174456  4242 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.189953  4125 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.214118  4261 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.279132  3770 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.279799  4261 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.179625 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.194001 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.297420 30988 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
W20260709 12:26:00.297467 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 117904us
I20260709 12:26:00.298422 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:00.299038 30988 raft_consensus.cc:2368] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20.
I20260709 12:26:00.194629 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:00.299527 30983 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:00.299571 31106 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.192423 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.195227 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.136503  4051 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.306630  4051 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:00.306957 30983 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:00.190299  4205 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:26:00.195753  3679 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:00.188511  4225 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.195991 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:00.432044 31033 connection.cc:256] RPC call timeout handler was delayed by -1.1146802040002513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
I20260709 12:26:00.440871 30987 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.186283  4206 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.455660  4206 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.196677  4264 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.457695  4264 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:00.000658 31144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2543) took 2181 ms (client timeout 2121 ms). Trace:
W20260709 12:26:00.458304 31144 rpcz_store.cc:269] 0709 12:25:57.819483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.819889 (+   406us) service_pool.cc:224] Handling call
0709 12:26:00.000389 (+2180500us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54000947d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.000534 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412538112}
I20260709 12:26:00.459739 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.460485 31144 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:26:00.461634 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.196475 30986 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:00.201620 30991 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:00.468555 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.468662 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.469271 30986 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:26:00.469666 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.180075 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:00.472651 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 314411us
I20260709 12:26:00.472537  3679 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.204339 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.474704  3679 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.475231  3679 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:00.475354 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 274664us
I20260709 12:26:00.475603  3679 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.476007  3679 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:00.209028 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.479029 31138 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:00.209715 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.483139 31142 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.484158 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.484833 31142 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:00.210855 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.213104 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.487843 31143 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 18: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.488201 30989 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:00.488611 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.186297  4197 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.490669  4197 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.491209 30991 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:00.219952  3581 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.493343  3581 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:00.229964 31064 rpcz_store.cc:269] 0709 12:25:58.464309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.464457 (+   148us) service_pool.cc:224] Handling call
0709 12:26:00.048004 (+1583547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.232275 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.494115 31139 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.494371 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.495747 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.497124  4037 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:00.497530  4037 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:00.497995  4037 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:00.234539  4265 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.499279 30991 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:00.083417 31696 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 1195) took 1646 ms (client timeout 2193 ms). Trace:
I20260709 12:26:00.470108 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:00.500458 31696 rpcz_store.cc:269] 0709 12:25:58.437054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.437212 (+   158us) service_pool.cc:224] Handling call
0709 12:26:00.083362 (+1646150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.231408 31061 rpcz_store.cc:276] 0709 12:25:58.391409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.410859 (+ 19450us) service_pool.cc:224] Handling call
0709 12:25:58.411170 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.080173 (+1669003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.080872 (+   699us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54001a6dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:00.081022 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3837036928}
I20260709 12:26:00.236594 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.196988  4163 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.229785  4193 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.105993 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.248116  3893 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.250317  4249 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:00.127369 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08001b8960 after lost signal to thread 30924
I20260709 12:26:00.160326  4198 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.163381 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2869) took 1697 ms. Trace:
I20260709 12:26:00.206313  3591 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.228199  4062 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.258870  4032 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:00.214993  4263 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.248418 31135 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:26:00.259994 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.238049  3853 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.249441  4122 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:00.124868  3891 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.275151  4203 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.277928 31693 rpcz_store.cc:276] 0709 12:25:58.471773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.472167 (+   394us) service_pool.cc:224] Handling call
0709 12:26:00.277384 (+1805217us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54005e0954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.277505 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234289920}
I20260709 12:26:00.291074  3770 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.292930  4261 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.296469 31141 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:00.195053 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.294929 30882 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 4 to 14, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 14 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:00.304852  4194 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:00.305506 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 113159us
I20260709 12:26:00.305598  4242 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:00.305650 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:00.307013  4051 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.191728  4005 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:00.299049 31060 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:00.300999 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.309105 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.246054  4115 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.312033 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.262612  4068 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.338953 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.343107 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:00.374017 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.225935 31691 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.501742 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.515828 31061 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.515923 31138 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:26:00.516454 31067 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
W20260709 12:26:00.516929 31067 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2871) took 2049 ms (client timeout 2179 ms). Trace:
I20260709 12:26:00.516973 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:00.517334 31067 rpcz_store.cc:269] 0709 12:25:58.467840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.105760 (+1637920us) service_pool.cc:224] Handling call
0709 12:26:00.516872 (+411112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.518081 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.518458  4051 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:00.518875 31691 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.519621 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.520179 30988 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:26:00.522011 31062 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.522421 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2603) took 2014 ms. Trace:
I20260709 12:26:00.522713 31062 rpcz_store.cc:276] 0709 12:25:58.508358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.232374 (+1724016us) service_pool.cc:224] Handling call
0709 12:26:00.522364 (+289990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.523262  4242 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.523691  4242 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:00.524241 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.522644 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.525439  4163 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.525920 30990 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:00.526266 30984 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:00.527417 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800244320 after lost signal to thread 31016
I20260709 12:26:00.527676 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.528224 31142 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
W20260709 12:26:00.528673 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08002ce140 after lost signal to thread 31093
I20260709 12:26:00.530366  4193 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.530705  4068 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.440275  4245 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.531410 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 230498us
I20260709 12:26:00.532169 31106 leader_election.cc:400] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:26:00.531857  4245 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:00.532749 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.532008 30882 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 13, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 13 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:00.529031 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.440354  4082 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.440268  4173 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.451082  4043 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.451210  4270 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.449926  4093 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.451994 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.453114  4079 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:00.453437  4054 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.471868 31065 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:00.206565 31694 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:00.232426 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:00.528764 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.515797 31068 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:00.533793 30882 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:00.431525  4105 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.532454 31106 leader_election.cc:403] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:00.440569  4175 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.530423  4005 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.534195  4245 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.534504  4082 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.561901  4082 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.534803  4043 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.563721 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.564253  4043 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.500851 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:00.536329 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.536332  4249 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.535015  4270 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:00.534927 31135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2547) took 2108 ms (client timeout 1926 ms). Trace:
W20260709 12:26:00.542500 31065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2867) took 2076 ms (client timeout 2087 ms). Trace:
I20260709 12:26:00.541326 31136 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.535804  4194 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.566871  3853 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.567019  4194 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.567337  3853 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.567515 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.567767  3853 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.568152  4270 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.544580 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.568888 31067 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
W20260709 12:26:00.566372 31135 rpcz_store.cc:269] 0709 12:25:58.426338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.428574 (+  2236us) service_pool.cc:224] Handling call
0709 12:26:00.534769 (+2106195us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b540006e0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.534860 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356460288}
I20260709 12:26:00.542752  4203 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:00.569989 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.570569  4203 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.570909 31135 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.570988  4203 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.571776 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.572321 31135 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:00.573107 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.573738 31135 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.522730  3891 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.547147  4032 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:00.517343  3591 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.548616 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.549652  4122 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.552652 31698 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:00.554584 31140 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.537931 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:00.557583 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.549289  4263 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.577605 31106 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.577976 31058 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:00.547395 31059 rpcz_store.cc:276] 0709 12:25:58.466249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.161826 (+1695577us) service_pool.cc:224] Handling call
0709 12:26:00.163326 (+  1500us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.534740  4173 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.579233 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.580031 31058 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:00.580587 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2872) took 2112 ms. Trace:
I20260709 12:26:00.580816  4173 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.535215  4093 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.535516 31138 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:00.564280  4054 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.551436 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:00.582839 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.535857 31142 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:00.583485 31062 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.556265  4175 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.545881 31141 raft_consensus.cc:2368] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:26:00.584110 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.584532 31062 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:00.585062 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1463) took 2108 ms. Trace:
I20260709 12:26:00.591718 31062 rpcz_store.cc:276] 0709 12:25:58.476259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.584042 (+2107783us) service_pool.cc:224] Handling call
0709 12:26:00.584275 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.584841 (+   566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.585014 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.592785 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:00.585122 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.558508 30983 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:00.568166 31136 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:00.594354 31142 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:00.584901 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.595322 31141 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.557933 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:00.542579 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90664.7us
I20260709 12:26:00.569991 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:00.546880 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.598498 31068 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:00.599502 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.564321 31060 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:00.600210 31068 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:00.601264 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.585899 31138 raft_consensus.cc:2410] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:26:00.578029  4263 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.586385  3591 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:00.575253  4032 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.556085  4105 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.583114  4093 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:00.566517 31065 rpcz_store.cc:269] 0709 12:25:58.465681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.053695 (+1588014us) service_pool.cc:224] Handling call
0709 12:26:00.542321 (+488626us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54000c7e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.542435 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555273088}
I20260709 12:26:00.575294  4122 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.549227 30882 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:00.585289  4175 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.626781  4163 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:00.627214  4163 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.535152  4265 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:00.549463 31664 connection.cc:256] RPC call timeout handler was delayed by -1.2357235870003933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260709 12:26:00.580915 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1462) took 2105 ms. Trace:
I20260709 12:26:00.588900 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.627795  4163 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:00.627926 31664 connection.cc:256] RPC call timeout handler was delayed by -1.1763235870002973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 12:26:00.628352 31664 connection.cc:256] RPC call timeout handler was delayed by -1.1628235870002754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
W20260709 12:26:00.628628 31664 connection.cc:256] RPC call timeout handler was delayed by -1.0296235870005148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 12:26:00.575773 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:00.628880 31664 connection.cc:256] RPC call timeout handler was delayed by -1.0043378640002629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
I20260709 12:26:00.629173 31061 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:00.582432 31064 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:00.630079 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:00.580892 31058 rpcz_store.cc:276] 0709 12:25:58.468152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.579429 (+2111277us) service_pool.cc:224] Handling call
0709 12:26:00.579748 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.580370 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.580540 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.554454  4062 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.437023  4076 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.608678  4057 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.630990 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.634665 31136 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:00.631074  4051 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.636166 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.636871 31136 raft_consensus.cc:2368] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:00.462163 31144 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:00.637547  4068 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:00.638537 31135 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:00.638954 31144 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:00.603868 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:00.639600 31062 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:00.640250 31144 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:00.640527  3808 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.602967  4263 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.620796  4043 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:00.619577  3591 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.655597 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.661748 31142 raft_consensus.cc:2410] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:00.621806 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.602311 30987 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:00.618641 31068 raft_consensus.cc:2410] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:00.662964 31060 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
W20260709 12:26:00.608880 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 53184.8us
I20260709 12:26:00.667075 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.554282 30984 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:00.624193 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.669147 31065 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:26:00.602936  4027 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:26:00.627235  4122 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:00.682488 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.679656 30955 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.636818  4051 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.639047 31136 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:26:00.605412 31067 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.631145 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.694952 31136 pending_rounds.cc:85] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 12
I20260709 12:26:00.695263 31136 pending_rounds.cc:107] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:26:00.642699 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:00.696058 31694 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 14: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:00.643123 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.631659 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.697021 31058 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:00.632930 31063 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:00.644471 31697 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:00.644093 31059 rpcz_store.cc:276] 0709 12:25:58.475927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.580208 (+2104281us) service_pool.cc:224] Handling call
0709 12:26:00.580474 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.580629 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.580652 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.580687 (+    35us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:00.580716 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.580763 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.580865 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.644941 31140 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
W20260709 12:26:00.645377 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.645480 31139 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
W20260709 12:26:00.529048 30921 debug-util.cc:398] Leaking SignalData structure 0x7b080032dfa0 after lost signal to thread 31094
I20260709 12:26:00.699347 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.700218 31139 pending_rounds.cc:85] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 8
W20260709 12:26:00.700366 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08003b8f00 after lost signal to thread 31168
I20260709 12:26:00.700510 31139 pending_rounds.cc:107] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 9.9
W20260709 12:26:00.566495 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 89261.2us
I20260709 12:26:00.646687 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.648216 31691 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:00.701239 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:00.700683 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800346700 after lost signal to thread 31169
I20260709 12:26:00.642540 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:00.649528 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.621639 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.552486  4115 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:00.651660 31690 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:00.544201  4198 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.701853 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.558209  4005 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.650548 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.678342 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:00.710451 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59979.1us
I20260709 12:26:00.712103 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:00.712915 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:00.713866 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:00.714838 30985 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.680940 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.715770 31135 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.687202 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.633773  4057 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.652272  4263 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.721458 31689 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:00.695474 31136 raft_consensus.cc:2889] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.722434 30989 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:00.695542 31061 raft_consensus.cc:2410] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 12:26:00.705526 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 83956.1us
I20260709 12:26:00.632710  4105 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.705440 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:00.684194 30955 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:00.701242 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54634.3us
I20260709 12:26:00.700803 31139 raft_consensus.cc:2889] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.701771 31141 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:00.706745 31698 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:00.706881 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08003b5920 after lost signal to thread 31722
W20260709 12:26:00.729293 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800398fc0 after lost signal to thread 31723
I20260709 12:26:00.633494  4076 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.603359  4203 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.694468  4164 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:00.662212  3808 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:00.707370 31062 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:00.710425  4205 raft_consensus.cc:2764] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:26:00.725092  4249 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:00.735982 31062 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:00.736716 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2604) took 2228 ms. Trace:
I20260709 12:26:00.707574 31695 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 14: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.737440 31135 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:26:00.707569 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.738914 31059 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.708740 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.714114 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.713241 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.656011  3591 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.716331 31692 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 10.
I20260709 12:26:00.756213 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2605) took 2247 ms. Trace:
I20260709 12:26:00.756677 31059 rpcz_store.cc:276] 0709 12:25:58.508837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.755286 (+2246449us) service_pool.cc:224] Handling call
0709 12:26:00.755628 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.755836 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.755872 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.755925 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:00.755971 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.756033 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.756147 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.627056 30882 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 12, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 12 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:00.720613  3891 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:00.758579  3891 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.759009  3891 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:00.720886  4176 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.762449  4176 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.762840  4176 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.766398  4037 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.766945 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2600) took 2259 ms. Trace:
I20260709 12:26:00.767540  4076 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.768476  4037 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:00.769838 31140 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2550) took 2342 ms (client timeout 1715 ms). Trace:
W20260709 12:26:00.770236 31140 rpcz_store.cc:269] 0709 12:25:58.427923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.428047 (+   124us) service_pool.cc:224] Handling call
0709 12:26:00.769620 (+2341573us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54001c61d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.769774 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123744512}
I20260709 12:26:00.771219 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.772254 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.772791 31140 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:00.773537 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.738687 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.774344  4203 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.774514  4176 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.775017 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.775588 31135 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:00.776113 31137 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:00.776422 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.777089 31135 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.776818  3891 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.723331 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.733919  4164 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.778541 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.778842  4164 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:00.778956 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55688us
I20260709 12:26:00.779358 31140 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.779889 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.780107 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.780710 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.781134 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.781539 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.781690 31140 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:00.781929  4057 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:00.682192  4038 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:00.783228 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.730161 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.704948  3893 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:26:00.733564 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.689746  4242 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.789459 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.791514 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.737133 31062 rpcz_store.cc:276] 0709 12:25:58.508597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.735393 (+2226796us) service_pool.cc:224] Handling call
0709 12:26:00.735758 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.736440 (+   682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.736655 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.748013 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.792652 31697 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:00.752532 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2602) took 2244 ms. Trace:
I20260709 12:26:00.793391 31061 rpcz_store.cc:276] 0709 12:25:58.508130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.729925 (+2221795us) service_pool.cc:224] Handling call
0709 12:26:00.751905 (+ 21980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.752120 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.752159 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.752215 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:00.752269 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.752336 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.752454 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.717870  4265 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.775717 30984 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:00.716377 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:00.782752 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.795437 31140 raft_consensus.cc:2368] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:26:00.796322 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.782954 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.797637 31137 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.783385 31064 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.767369 31060 rpcz_store.cc:276] 0709 12:25:58.507616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.727286 (+2219670us) service_pool.cc:224] Handling call
0709 12:26:00.766508 (+ 39222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.766665 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.766689 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.766725 (+    36us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:00.766761 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.766808 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.766895 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.725750 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.727674 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:00.799819 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 72244.6us
I20260709 12:26:00.722609 31109 leader_election.cc:400] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20.
I20260709 12:26:00.801610 31109 leader_election.cc:403] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:26:00.784348 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.731770  4005 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.824525  4194 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.804224 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.825470  4037 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:00.825688 31696 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:00.825867  4037 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:00.804857 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.826288  4037 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:00.827502  4176 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.828882  3891 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:00.806775 31140 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.800890 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.845520 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.843915 31063 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:00.786936 31141 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:00.846586 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.808116  3853 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.808462  4068 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.810338  4175 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:00.810879 31689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 1192) took 2674 ms (client timeout 1570 ms). Trace:
I20260709 12:26:00.789728  4043 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:00.805718  4076 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:00.822515  4093 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:00.823432 30882 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 15, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 15 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:00.785167  4038 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:00.790813  4105 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:00.825120  4005 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:00.829998 31698 raft_consensus.cc:2410] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 12:26:00.831115  4261 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:00.788847 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.831624 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2607) took 2322 ms. Trace:
I20260709 12:26:00.883486 31135 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.883544 31062 rpcz_store.cc:276] 0709 12:25:58.509309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.830833 (+2321524us) service_pool.cc:224] Handling call
0709 12:26:00.831156 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.831328 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.831354 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.831391 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:00.831423 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.831469 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.831571 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.782277  4249 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:00.805356  4265 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.866533  4068 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:00.836845 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2613) took 2295 ms. Trace:
I20260709 12:26:00.753908  4204 raft_consensus.cc:2764] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:26:00.838892 31143 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.839977  4242 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:00.841094 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 112566us
I20260709 12:26:00.893033 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:00.842680 31064 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:00.843297 31065 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:00.804576 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:00.806378 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 88202.1us
I20260709 12:26:00.834133 31068 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:00.843271 31061 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:00.895363 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2608) took 2385 ms. Trace:
I20260709 12:26:00.895687 31068 rpcz_store.cc:276] 0709 12:25:58.509566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.833674 (+2324108us) service_pool.cc:224] Handling call
0709 12:26:00.833951 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.895157 (+ 61206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.895313 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:00.850322 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.896976 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 92454.2us
I20260709 12:26:00.873050  4076 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:00.862298 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61401.3us
I20260709 12:26:00.910542 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2615) took 2368 ms. Trace:
I20260709 12:26:00.910951 31065 rpcz_store.cc:276] 0709 12:25:58.542589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.837677 (+2295088us) service_pool.cc:224] Handling call
0709 12:26:00.843154 (+  5477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.894720 (+ 51566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.910293 (+ 15573us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b54000c7950. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.910456 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11959168}
I20260709 12:26:00.863153 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.865137 30986 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:00.865792 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:00.914041 30987 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.914798 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.805574  4203 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.868960 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.869418 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.869845 31138 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:00.867503 31689 rpcz_store.cc:269] 0709 12:25:58.136801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.136938 (+   137us) service_pool.cc:224] Handling call
0709 12:26:00.810717 (+2673779us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b54006a1654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.810816 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5867668864}
I20260709 12:26:00.870101  4043 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.872284 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:00.874068 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.929917 31141 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.872452 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:00.875504 31063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2610) took 2365 ms (client timeout 2505 ms). Trace:
I20260709 12:26:00.930541 31137 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:26:00.930486 31063 rpcz_store.cc:269] 0709 12:25:58.510075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.232221 (+1722146us) service_pool.cc:224] Handling call
0709 12:26:00.875303 (+643082us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b54000c6054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:00.875434 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211952000}
I20260709 12:26:00.931645 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.932548 31137 raft_consensus.cc:2368] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:00.933408 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.876626 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.934703 31136 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.933974 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.935250 31142 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:00.935621 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.936318 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.937242 31142 raft_consensus.cc:2368] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:00.878841 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.809969  4263 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.882706 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:00.884895 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:00.887691 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 194150us
I20260709 12:26:00.940279 30992 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:00.870350  4176 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:00.941126  4176 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.941601  4176 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:00.943097 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.887861 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.946262 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:00.946866 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.947126  4054 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.947678  4054 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.948084  4054 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.950955 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.952518 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.888651 31058 rpcz_store.cc:276] 0709 12:25:58.541828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.836139 (+2294311us) service_pool.cc:224] Handling call
0709 12:26:00.836406 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.836573 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.836598 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.836637 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:00.836672 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.836720 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.836801 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.955175 31142 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:00.870867  4175 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:00.955839  4175 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:00.789638  4184 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.047s	sys 0.086s
W20260709 12:26:00.958513  4184 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.047s	sys 0.086s
I20260709 12:26:00.959390 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.897610 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.861617 30985 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:00.883229  4005 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:00.960726 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 63178.6us
I20260709 12:26:00.961546 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:00.881579  4038 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:00.962416 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:00.963563  4245 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:00.964023  4245 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:00.964815  4245 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.966382  4079 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:00.966882  4079 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:00.967674  4079 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.969969  3770 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:00.916205 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.916931 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.972263 31143 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 10.
I20260709 12:26:00.973284 31143 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:00.974753  4264 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:00.937028 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.976183 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:00.976616 31136 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:00.977253  3770 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:00.938503 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:00.939657 30988 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.978421  3770 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.836087 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2611) took 2325 ms. Trace:
I20260709 12:26:00.979744 31060 rpcz_store.cc:276] 0709 12:25:58.533962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.835351 (+2301389us) service_pool.cc:224] Handling call
0709 12:26:00.835643 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.835808 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.835833 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.835872 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:00.835909 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.835956 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.836039 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.987226  4038 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:00.989173 31136 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:00.992748  4005 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:00.996351 31142 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:00.999574  4038 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:01.005861  4005 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:01.008167 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 63425.1us
I20260709 12:26:01.010890  4225 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:01.011866 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:01.012984  4100 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.016498 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:01.017318 30988 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:26:00.888273  4265 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.864439  3679 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:01.022039 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 94449.4us
I20260709 12:26:01.023463 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:01.024044 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:01.024546 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:01.036274 31141 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:01.028964 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:00.927268 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:01.037179 31140 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:01.036290 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.038551 31138 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 18: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:00.891176  4105 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:01.039691 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.039574 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:01.059763 31138 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:26:01.056201 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 129011us
I20260709 12:26:01.061914 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:01.039638 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:01.063844 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:01.064602 31137 raft_consensus.cc:2376] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13. Re-sending same reply.
I20260709 12:26:01.056973 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:00.920471  4076 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.919497  4093 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:01.069592 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:01.082304 31138 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 12: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:01.065589 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.085039 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:01.066526 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 90435.5us
I20260709 12:26:00.919060  3808 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:01.055706 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.092741 31140 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:01.023145  4043 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:01.075624 30987 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:01.041011 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:01.087256 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:01.100653 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.101419 31138 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:01.087771 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:01.091403 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 51906.3us
I20260709 12:26:00.918442  4203 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.096166 31140 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:01.048213 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.055224  4005 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.105067 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:01.110188 30990 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:01.114878 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54308.6us
I20260709 12:26:00.904809  4057 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.174s
I20260709 12:26:01.115833 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:01.123312 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:01.038546  4038 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:26:00.918197  4122 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.175s
I20260709 12:26:01.078949  4197 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:01.092048  3581 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:01.091038  4242 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.029986  3891 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:26:01.092059  3808 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:01.107486  4062 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:01.091181  4284 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:01.034319  4193 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 12:26:01.097426  4093 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:01.134832  4115 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:01.106729  3913 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:01.117913  4203 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.056521  4037 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:01.150889  3770 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:01.190822 31068 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:01.126151  4204 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:01.067365  4164 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:01.206262 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 285393us
I20260709 12:26:01.215370  3808 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.117571  4163 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:01.132085  4205 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:01.137718  4286 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.057771 30882 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:01.230063  4115 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:01.143291  3581 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:01.148106  4099 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:01.217653 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 103597us
I20260709 12:26:01.268260 30983 raft_consensus.cc:2368] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20.
I20260709 12:26:01.125756  4005 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:01.252940 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 137209us
I20260709 12:26:01.211674  4062 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.254617 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:01.224629  4051 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:01.230947  4093 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.231747  4284 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:01.265164  4115 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.239740  4184 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:01.238615  3358 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.238175  3853 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:01.228430  4105 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:01.250784  4204 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.224426  3893 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:26:01.192247  4242 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:01.270514 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:01.257464  4205 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:01.136523  4249 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:01.238860  4054 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:01.263161  4194 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:01.355734 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:01.231634  3913 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.267491  3581 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:01.304140 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 174674us
I20260709 12:26:01.306509  4038 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:01.307938  4005 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:01.308207  4284 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:01.256088  3846 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:26:01.327548  4205 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:01.295671  4184 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:01.275456  4062 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.309005  4164 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:01.264554  3591 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:01.318120 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47722.5us
I20260709 12:26:01.379611 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:01.380273  3591 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:01.320196  4282 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:01.380977 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:01.320442  4193 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:01.323311  4276 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 12:26:01.323118 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2385) took 1027 ms. Trace:
I20260709 12:26:01.382203  4193 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:01.382702  4193 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.382387 31139 rpcz_store.cc:276] 0709 12:26:00.295500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.699733 (+404233us) service_pool.cc:224] Handling call
0709 12:26:00.700027 (+   294us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:00.732031 (+ 32004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.732064 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:00.732615 (+   551us) log.cc:844] Serialized 78 byte log entry
0709 12:26:01.047700 (+315085us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:01.047736 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.047796 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.322815 (+275019us) raft_consensus.cc:1685] finished
0709 12:26:01.322844 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.323043 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":207,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":17936,"thread_start_us":239971,"threads_started":1,"wal-append.queue_time_us":486075}
I20260709 12:26:01.311623  3358 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:01.386801  4287 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:26:01.386905  3358 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:01.308789 30882 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 16, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 16 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:01.388700 30882 catalog_manager.cc:5697] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 11, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 11 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
W20260709 12:26:01.344473 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 89930.5us
I20260709 12:26:01.392160 30882 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 16, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 16 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:01.394112 30882 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:01.395864 30882 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 14, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 14 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:01.354928  4100 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:01.399435  4270 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:01.399832  4270 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:01.400609  4270 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:01.401702 31144 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:01.402529 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1061) took 1095 ms. Trace:
I20260709 12:26:01.377101  4038 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:01.402894 30991 rpcz_store.cc:276] 0709 12:26:00.306872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.500189 (+193317us) service_pool.cc:224] Handling call
0709 12:26:00.500551 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.500987 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:00.501350 (+   363us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.877513 (+376163us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.885673 (+  8160us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.885808 (+   135us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:00.885838 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.885901 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.402312 (+516411us) raft_consensus.cc:1685] finished
0709 12:26:01.402334 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.402469 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15589,"prepare.queue_time_us":344,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":15954,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":1270,"spinlock_wait_cycles":10112,"thread_start_us":359422,"threads_started":1,"wal-append.queue_time_us":391176}
I20260709 12:26:01.404131  4038 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:01.347275 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.314960  4204 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.356874  3679 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:01.359900  3913 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:01.323153  4037 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:01.351686  4242 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:01.352803 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 84272.3us
I20260709 12:26:01.310915  4051 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:26:01.359535 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 318628us
I20260709 12:26:01.411055 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:01.411111  4051 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:01.308861  4261 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20.
I20260709 12:26:01.360141 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:01.420107 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:01.421916  4300 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:26:01.398011 30882 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 5 to 10, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 10 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:01.423475  4284 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:01.424990  4037 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.425460  4037 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:01.426482 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:01.426908 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.427500 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.427611 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.427953 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.428501 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.428678 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.428972 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:01.423422  3358 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:26:01.431869 31137 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
W20260709 12:26:01.432693 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 309437us
W20260709 12:26:01.432796 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 72754.4us
I20260709 12:26:01.434437 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:01.437189 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.438531 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.439240 31140 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:01.442037 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.442783 31140 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:26:01.445082 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.447495 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.448338 31135 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:01.448931 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.449090 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:01.449510 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.450040 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.451125 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.488015 31695 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:01.487932 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.488262 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:01.507833 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 77266us
I20260709 12:26:01.526052 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.527029 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:01.528131 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.417870  4100 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:01.508886 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:01.509582 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52527.6us
I20260709 12:26:01.511636  4037 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:01.429867  4204 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:01.448637  4197 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:01.453112  4082 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:01.516660  4304 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:01.507336  4261 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:01.520826  4076 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:01.523165 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.523962 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.380973  3591 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:01.527107  3358 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:01.421262  4115 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:01.534706 31697 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:01.549878 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.550834 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.555889 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:01.591917 31060 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:01.531296  4284 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:01.516110 30882 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 13, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 13 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:01.567821 31698 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:01.555246  4293 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:01.521538  4193 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:01.572479 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:01.572927 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 141583us
I20260709 12:26:01.574112 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:01.550357  4068 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.645s	user 0.003s	sys 0.007s
I20260709 12:26:01.631804 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.613426 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.633679 31692 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:01.614156 31067 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:01.635079 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:01.631695 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:01.566229  4122 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:01.640682 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:01.641237 31060 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:01.642194 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.646677 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.596068  4284 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:02.080335 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 553177us
I20260709 12:26:02.082500  4284 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.085000  4284 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:02.010310 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.011533 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 365104us
I20260709 12:26:02.087659 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 2892) took 1467 ms. Trace:
I20260709 12:26:02.088161 31067 rpcz_store.cc:276] 0709 12:26:00.620135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.716066 (+ 95931us) service_pool.cc:224] Handling call
0709 12:26:02.087086 (+1371020us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54000c6554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:02.087405 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1884028672}
W20260709 12:26:02.012106 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 371408us
W20260709 12:26:02.074002 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.009s	sys 0.021s
W20260709 12:26:02.091116 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.009s	sys 0.021s
W20260709 12:26:02.093647 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 605398us
I20260709 12:26:02.085829  4193 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:02.095072 31065 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:02.013875 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:02.082060 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:01.988592 30882 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 11, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 11 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:26:02.098680  4176 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.156s	user 0.010s	sys 0.011s
I20260709 12:26:02.099443  4176 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.099795 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.099680 30882 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 11, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 11 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.101071  4076 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:01.595377 31139 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
W20260709 12:26:02.102524  4297 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.561s	user 0.004s	sys 0.009s
I20260709 12:26:01.629073 31693 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:01.596513 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.532647 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.612780 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:01.600304  3591 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:02.107358  4304 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:02.108419  3591 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:01.642477  3679 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:02.110142 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:01.637981  4068 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.112303 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:01.522775  4099 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:26:01.508121  4125 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:26:01.599812  4163 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:01.565235  4175 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.000s
I20260709 12:26:01.524731  4184 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.137666  4175 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.138741  4184 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.139334 31136 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:02.141505  4175 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:01.666699  4173 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.055s
W20260709 12:26:02.143074 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49057.6us
I20260709 12:26:01.596031  4203 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.148974  4203 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:01.596683  4242 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:02.150309  4176 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:01.643280  4283 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:02.053681  4082 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:02.151903  4082 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:02.055931  4054 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.075235 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:02.152392  4082 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:02.076083  4115 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.157886  4125 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:02.158481  4125 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.158365 30882 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 12, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 12 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.160005  4125 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.080123  4204 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.078658  4263 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:02.163359  4204 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.163671 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:02.164197  4204 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.165153  4283 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:02.165759  4283 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:01.629705 31142 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:02.113821  4068 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:02.135084 30983 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 1067) took 1422 ms (client timeout 1569 ms). Trace:
I20260709 12:26:02.167248  4283 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:02.168052 30983 rpcz_store.cc:269] 0709 12:26:00.712584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.712712 (+   128us) service_pool.cc:224] Handling call
0709 12:26:02.127277 (+1414565us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b5400226854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:02.134824 (+  7547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275526400}
I20260709 12:26:02.134862 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2390) took 1504 ms. Trace:
I20260709 12:26:02.160821  4313 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.152381 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.169124 31143 rpcz_store.cc:276] 0709 12:26:00.630567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.973942 (+343375us) service_pool.cc:224] Handling call
0709 12:26:00.974201 (+   259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.055700 (+ 81499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.055746 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.081345 (+ 25599us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.310439 (+229094us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:01.310474 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.310534 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.126125 (+815591us) raft_consensus.cc:1685] finished
0709 12:26:02.126175 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.134582 (+  8407us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53588,"raft.queue_time_us":79381,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":31653,"thread_start_us":166880,"threads_started":1,"wal-append.queue_time_us":251671}
I20260709 12:26:02.171557 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.172279 31143 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:26:02.172343 31060 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:02.175549 31143 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:26:02.156383 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.181639 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2407) took 1411 ms. Trace:
I20260709 12:26:02.182174 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.184350 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1035) took 1890 ms. Trace:
I20260709 12:26:02.184474 31067 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:26:02.184932 31067 pending_rounds.cc:85] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 10
I20260709 12:26:02.185307 31067 pending_rounds.cc:107] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:26:02.184801 31144 rpcz_store.cc:276] 0709 12:26:00.294242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.640893 (+346651us) service_pool.cc:224] Handling call
0709 12:26:00.641121 (+   228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.402098 (+760977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.183802 (+781704us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b5400062f54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:02.184161 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1748769024}
I20260709 12:26:02.187135 31691 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:26:02.187884 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.182035 31135 rpcz_store.cc:276] 0709 12:26:00.770042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.945973 (+175931us) service_pool.cc:224] Handling call
0709 12:26:02.181194 (+1235221us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b540006d6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:02.181404 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088916864}
I20260709 12:26:02.189495 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1212) took 1106 ms. Trace:
I20260709 12:26:02.190197 31696 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:02.190573  4115 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:02.189522 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 1213) took 1106 ms. Trace:
I20260709 12:26:02.191169  4115 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:02.191670  4115 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:02.191453 31697 rpcz_store.cc:276] 0709 12:26:01.082730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.084747 (+  2017us) service_pool.cc:224] Handling call
0709 12:26:02.189098 (+1104351us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54004d7e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:02.189280 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724174208}
I20260709 12:26:02.185810 31067 raft_consensus.cc:2889] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.195192 30958 leader_election.cc:400] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13.
I20260709 12:26:02.198983 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2617) took 2034 ms. Trace:
I20260709 12:26:02.189982 31689 rpcz_store.cc:276] 0709 12:26:01.082422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.082577 (+   155us) service_pool.cc:224] Handling call
0709 12:26:01.122740 (+ 40163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.310042 (+187302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.310069 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.310108 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:01.310140 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.188926 (+878786us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400801bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:02.188993 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.189233 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430223616}
I20260709 12:26:02.199455 31061 rpcz_store.cc:276] 0709 12:26:00.164224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.929291 (+765067us) service_pool.cc:224] Handling call
0709 12:26:00.929684 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.929916 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:01.064913 (+134997us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.178785 (+1113872us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b18002f7be0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287
0709 12:26:02.179027 (+   242us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.179155 (+   128us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:02.179323 (+   168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.198489 (+ 19166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.198720 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93583,"prepare.queue_time_us":168393,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":54398,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":14225,"spinlock_wait_cycles":11267456,"thread_start_us":190105,"threads_started":2,"wal-append.queue_time_us":173760}
I20260709 12:26:02.202155  4005 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:02.202711  4005 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:02.204918  4093 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.908s	user 0.009s	sys 0.009s
I20260709 12:26:02.200459 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.205853  4005 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.209236  4005 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.209470  4204 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.209702  4198 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:02.210206  4198 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:02.211660  4198 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.214072 30958 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:02.214953 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.216784  4176 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:02.180279 30882 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 12, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 12 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:02.217331  4176 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:02.217684  4279 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.954s	user 0.000s	sys 0.003s
I20260709 12:26:02.217901  4176 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.219215 30882 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 14, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 14 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.226287  4204 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.226398  4093 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.229117  4093 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.232720 31063 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:02.234866 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2627) took 1735 ms. Trace:
I20260709 12:26:02.235291 31063 rpcz_store.cc:276] 0709 12:26:00.499340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.080515 (+581175us) service_pool.cc:224] Handling call
0709 12:26:01.080784 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.233165 (+1152381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.234560 (+  1395us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001a77d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:02.234798 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2648273280}
I20260709 12:26:02.236994  4176 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.237265 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2616) took 2073 ms. Trace:
I20260709 12:26:02.237179  4100 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:02.237816 31065 rpcz_store.cc:276] 0709 12:26:00.163897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.928545 (+764648us) service_pool.cc:224] Handling call
0709 12:26:02.094442 (+1165897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.095589 (+  1147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.237090 (+141501us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000c6550. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:02.237215 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2679837440}
I20260709 12:26:02.238941  4100 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:02.227347 30958 leader_election.cc:400] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:02.239472  4100 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.241086  4100 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.243178 31061 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:02.246730  4198 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:02.249866  3581 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.877s	user 0.008s	sys 0.013s
I20260709 12:26:02.252795 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.253727 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2393) took 1622 ms. Trace:
I20260709 12:26:02.254189 31136 rpcz_store.cc:276] 0709 12:26:00.631365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.991421 (+360056us) service_pool.cc:224] Handling call
0709 12:26:00.991746 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.140069 (+1148323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.253485 (+113416us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003f48d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:02.253668 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2582064896}
I20260709 12:26:02.252862 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:02.257157  4176 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.257288 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.258306  4290 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:02.260317  4293 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.628s	user 0.010s	sys 0.010s
I20260709 12:26:02.260579  4163 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:02.262864  4317 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
W20260709 12:26:02.263185  4205 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.845s	user 0.014s	sys 0.007s
I20260709 12:26:02.266608 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.268530 31060 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:02.270777  4032 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:26:02.225904  4043 consensus_meta.cc:287] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.101s	user 0.003s	sys 0.023s
W20260709 12:26:02.233685  3913 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.825s	user 0.004s	sys 0.014s
W20260709 12:26:02.322427 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.310958  4163 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:02.329263 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:02.330610 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:02.336706  4286 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.069s	user 0.017s	sys 0.011s
I20260709 12:26:02.337244  4286 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:02.328924  3913 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.327504  4043 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.338174  4057 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:02.338339  4265 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.225s	user 0.010s	sys 0.015s
I20260709 12:26:02.343401 31065 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:02.343679  4265 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.344110  4043 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.346485 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.346668  4100 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.347283 31065 raft_consensus.cc:2368] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:02.336733  4270 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.350744 31064 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:02.352319 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90085.5us
I20260709 12:26:02.352368  4318 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:02.352667 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1468) took 2381 ms. Trace:
I20260709 12:26:02.353413 30958 leader_election.cc:400] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:02.353802 31059 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:02.354349  4176 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:02.355252 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2606) took 3846 ms. Trace:
I20260709 12:26:02.355794  4176 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:02.356762  4176 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:02.356817  3853 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:02.346030  4265 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.357357  4176 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:02.358206  4302 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:02.358654  3808 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.051s	user 0.012s	sys 0.010s
I20260709 12:26:02.358809  4302 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:02.359264  3808 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.353665 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:02.359828  4302 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.361200 30955 leader_election.cc:400] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:02.361347  4302 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.361601 30955 leader_election.cc:403] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:26:02.357962  3679 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:02.362566  4292 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:02.362802  3679 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:02.363483  3808 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.341833  3913 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.364517 31068 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:02.364125  3679 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:02.366752 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 109552us
I20260709 12:26:02.367475  4164 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.368444 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:02.368685  4225 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:02.369501 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.369351  4225 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.366593  4303 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:02.372246  4292 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:02.373251  4292 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.374871  4305 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:02.375329  4305 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.375689  4305 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.376786  4122 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.377182 31058 raft_consensus.cc:2410] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:26:02.358247 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.003s	sys 0.017s
W20260709 12:26:02.378820 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.004s	sys 0.017s
I20260709 12:26:02.379737  4057 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:26:02.381119  4315 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.227s	user 0.000s	sys 0.003s
I20260709 12:26:02.221429 30882 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 14, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 14 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:02.353137 31064 rpcz_store.cc:276] 0709 12:25:59.970953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.906402 (+935449us) service_pool.cc:224] Handling call
0709 12:26:00.906769 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.351196 (+1444427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.352417 (+  1221us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54001bb554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:02.352609 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3319926656}
I20260709 12:26:02.386183 31060 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:02.368628  4204 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:02.365976  3679 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.323428  4032 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.355641 31059 rpcz_store.cc:276] 0709 12:25:58.509064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.757883 (+2248819us) service_pool.cc:224] Handling call
0709 12:26:00.758204 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.354318 (+1596114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.354992 (+   674us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400268c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:02.355192 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3668452224}
W20260709 12:26:02.392323 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 293030us
I20260709 12:26:02.324841  4293 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.334028  4163 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.350284  4125 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.401338 31067 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:02.313349  3581 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.404894 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.324695  4205 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.401575  4313 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:02.425665 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:02.391680  4318 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:02.433022 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 102553us
I20260709 12:26:02.431828  4204 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:02.434093  4204 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.434511  4204 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.435424  4225 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:02.434093 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 64701.2us
I20260709 12:26:02.359094  4176 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.451647 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2638) took 1031 ms. Trace:
I20260709 12:26:02.454716 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:02.452060 31058 rpcz_store.cc:276] 0709 12:26:01.417066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.428772 (+ 11706us) service_pool.cc:224] Handling call
0709 12:26:02.441708 (+1012936us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b54000c7454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:02.448620 (+  6912us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2178146048}
I20260709 12:26:02.472054 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:02.472427  4062 consensus_meta.cc:287] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.984s	user 0.014s	sys 0.011s
I20260709 12:26:02.487635  4062 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.490396  4225 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:02.491690  4321 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:02.491982  4225 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:02.492396  4321 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.492589  4225 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.492970  4321 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:02.494396 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:02.494871 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:02.495275 31068 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
W20260709 12:26:02.496224 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 100467us
I20260709 12:26:02.497764 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2566) took 1060 ms. Trace:
I20260709 12:26:02.498204 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.498155 31137 rpcz_store.cc:276] 0709 12:26:01.436826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.436972 (+   146us) service_pool.cc:224] Handling call
0709 12:26:02.497382 (+1060410us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b540029e5d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:02.497575 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2283511168}
I20260709 12:26:02.500360 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:02.501423 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 56176.3us
I20260709 12:26:02.489969  4062 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:02.473268  4316 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:02.496224 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:02.511312  3893 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:02.511886  4313 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:02.511956  3893 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.478384 31140 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:02.432616  4082 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.516219  4082 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.473569 31142 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:02.516745  4032 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.517978  4032 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.462168  4051 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:26:02.486361 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:02.472575 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.519731  3893 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.503269 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:02.505764 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.448877  4318 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.507037 30985 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:02.504336 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:02.507408 30882 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 13, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 13 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
W20260709 12:26:02.511504 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.485250  4176 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.502019 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.502326 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.533623 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.534281 30991 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:02.535562 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 87636.4us
I20260709 12:26:02.537251  3853 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:02.537489 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:02.537767  3853 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.538262  3853 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.539057 30992 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:02.542204  3581 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:02.543661 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.543995  4122 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:26:02.545099  4122 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:02.545485  4122 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.546211 30987 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:02.547284  4122 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.548913  4122 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.548163  4245 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:02.551162 30989 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:26:02.554389 30990 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.552032  4245 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.548681  3853 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.550393  4163 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.557057  3853 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.558593  3581 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:02.559693 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.556527  4245 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:02.560439 30882 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 12, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 12 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:02.559113 30991 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:02.569618 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:02.569891 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.571873  4163 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:02.552719 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 50528.1us
I20260709 12:26:02.577567 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.579038 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:02.579814  3591 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:02.580169  3581 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:02.580739  3913 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.581759 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.493889  4225 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:02.583209 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45798.9us
I20260709 12:26:02.585348 30984 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:02.489989  4205 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:02.482963  4198 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 12:26:02.587455 30988 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:02.587872  4198 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:02.588371  4198 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:02.588529  4205 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:02.593109  4164 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
W20260709 12:26:02.593943 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.597152 30986 raft_consensus.cc:2410] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 17 because replica has last-logged OpId of term: 16 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 12:26:02.604275 30990 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:02.522601  4318 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:02.606503 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.608930 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.614193 31060 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 14: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:02.598975  4082 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:02.624464 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.625041 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:02.625200  4324 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:26:02.625658 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:02.599445  3913 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:26:02.594558  4303 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:26:02.636039 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.637503 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:02.637921 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 137679us
I20260709 12:26:02.639868 31065 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:02.643006 31059 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:02.643101 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.644030 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:02.642241 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:02.645829 31059 raft_consensus.cc:2368] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19.
I20260709 12:26:02.646747 30990 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:02.647002 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 65393.4us
I20260709 12:26:02.635000  4163 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:02.649483  4163 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.652406  4163 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:02.652928 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.655156  4204 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.656570 31106 leader_election.cc:400] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19.
I20260709 12:26:02.656884 31106 leader_election.cc:403] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:26:02.657402  4204 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.657068  4245 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.659343  4245 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.661319 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:02.662488 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:02.663071 31140 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:02.583397 30882 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 11, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 11 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:02.665143  4307 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:02.667097 30882 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 13, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 13 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:02.660712  4115 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.676829 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1472) took 2396 ms. Trace:
W20260709 12:26:02.711782 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57821.9us
I20260709 12:26:02.695358  4225 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:02.715137 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.715986 30991 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:02.715883 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:02.715296 30987 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:02.712728 31062 rpcz_store.cc:276] 0709 12:26:00.204202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.039769 (+835567us) service_pool.cc:224] Handling call
0709 12:26:01.040032 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.115604 (+ 75572us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:02.240244 (+1124640us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b180040fa80. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf
0709 12:26:02.240499 (+   255us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.444626 (+204127us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.515598 (+ 70972us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b180040fa80. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:02.532146 (+ 16548us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.532307 (+   161us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:02.532352 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.585231 (+ 52879us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.585279 (+    48us) raft_consensus.cc:1685] finished
0709 12:26:02.585292 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.600116 (+ 14824us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84190,"prepare.queue_time_us":75497,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":3969,"raft.queue_time_us":108031,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":11280,"spinlock_wait_cycles":17001088,"thread_start_us":274577,"threads_started":2,"wal-append.queue_time_us":208183}
I20260709 12:26:02.719569 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:02.719736  4125 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:02.721046  4100 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:02.721197  4115 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.722282 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:02.724355 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.722875 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:02.728693 31693 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:02.723181 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:02.731302 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:02.729892 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.738515 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:02.739697 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.743269 30989 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:02.743705 31696 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:02.730967  4100 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.740525  4125 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.744702  4307 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19.
I20260709 12:26:02.746193  4125 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:02.746270  4100 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.746548 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:02.747045 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:02.749527 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.750552 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:02.744719 31136 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:02.752281 31063 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:02.752631 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:02.753038 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:02.753962 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.754014 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:02.755309 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:02.755630  4318 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:02.755903  4176 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:02.754060 31140 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:02.758358  4125 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:02.755545 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.763711  4307 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:02.764035  4262 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:02.764114  4307 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:02.764503  4262 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.764638  4307 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:02.765465  4262 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.770736 31664 leader_election.cc:400] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20.
I20260709 12:26:02.772704 31664 leader_election.cc:403] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:26:02.774673  4262 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:02.779142  4284 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:02.780236  4284 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:02.781231  4284 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:02.781577  4284 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.782373  4054 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:02.782784  4054 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.775463 31137 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:02.783773  4054 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.784197 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:02.760255  4245 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:02.778468  4100 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:02.782801 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:02.779346  4082 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:02.726702  4305 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.787190  4054 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.787539 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.792172 31689 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:02.800482 31065 raft_consensus.cc:2393] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 9: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:02.799409 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:02.800741  4082 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:02.805543 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 80510.9us
I20260709 12:26:02.808861 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.810143 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:02.804639  4284 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:02.815411  4330 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:26:02.815658  4284 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.815902 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.817199 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:02.817725  4305 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.781572  4198 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:02.814702 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:02.818423  3679 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:02.818440  4198 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.822525 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.766609  4321 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.828138 31693 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:02.809933 30986 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:02.798812  4245 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:02.805006  3913 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.810258  4176 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:02.844947 31062 raft_consensus.cc:2368] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16.
W20260709 12:26:02.846633 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 91154.4us
I20260709 12:26:02.847863  4242 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:02.838155  4245 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:02.849872  4057 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:02.850299  4057 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:02.850710  4057 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:02.854418  4125 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:02.855173  4125 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:02.856918  4125 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.857396  4125 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.861166  4307 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.862550 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.863288 30989 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:02.863632  3913 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.865689  3913 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.847949 31063 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:02.834697  4163 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:02.869554 31697 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:02.872936  4027 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:02.806800  4115 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:26:02.879415  4321 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.845425  4176 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:02.889595  4318 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:02.890530 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 75974us
I20260709 12:26:02.891204  4242 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:02.894335 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:02.880007 31067 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:02.837270  4164 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:26:02.773471 30882 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:02.868462  4305 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:02.912712  4125 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:02.913210  4245 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.905014  4283 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:02.913647  4027 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:02.906606  4176 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:02.928745 31696 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:02.906544  4225 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:26:02.913210  4115 consensus_queue.cc:1243] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:02.926437  4198 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:02.948243 31058 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:02.926745  4283 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:02.928587  4027 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:02.956323 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:02.909252  4057 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:02.957331 31137 raft_consensus.cc:2393] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 11: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:26:02.960403 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 66187.2us
W20260709 12:26:02.960714 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 144900us
I20260709 12:26:02.961673 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:02.965893 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:02.961845 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:02.966121 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:02.970597 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:02.971447 31141 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:02.974850  4057 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:02.975193  4245 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:02.977571 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:02.979312 31140 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:02.982743 31142 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:02.986943 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:02.976471  4032 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:26:02.975941  4082 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.008494 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:03.009490  4332 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:03.010032 31661 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:03.011746  4339 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:03.012213  4339 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:03.012580  4339 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.013113  4054 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:03.014138  4290 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:03.005179  4283 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:03.016532 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:03.017495 31140 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:03.022564 31661 leader_election.cc:400] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16.
I20260709 12:26:03.025478 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:03.025930  4225 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:03.026979 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:03.027165 31696 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:03.027727 31065 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:03.028070 31696 raft_consensus.cc:606] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.028530 31696 raft_consensus.cc:614] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:03.021824 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:03.030568 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:03.031872  3893 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:03.032140 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.032682  3893 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:03.033805  4205 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:03.034487  4205 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:03.035706  4205 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.037904  4242 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:03.038632  4242 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:03.040436 31065 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:03.041699 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:03.042500  4242 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:03.044654 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:03.043278  4225 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:03.047238  4225 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:03.048382  4225 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.051121 31696 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:03.051853 31696 raft_consensus.cc:606] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:03.052201 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.052258 31696 raft_consensus.cc:614] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:03.054462 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49482.2us
I20260709 12:26:02.999675  4340 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:03.016067  4307 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:03.009526  4082 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:03.021083  3581 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:02.962662 30882 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 5 to 14, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 14 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:03.089206 31696 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:03.068768  3581 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:03.105216  4336 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:03.114462 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:03.115465  4082 raft_consensus.cc:2764] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:03.117180 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:03.118176  4082 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:03.054854  4027 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.127153  4027 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:03.150768 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 118782us
I20260709 12:26:03.092341  4100 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20.
I20260709 12:26:03.166909 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.168046 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:03.168730  4057 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:03.169121  4057 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.169454  4057 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.170425  4305 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:03.170969  4305 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:03.171414  4284 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:03.171911  4284 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:03.171840  4305 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.173115  4284 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.177491  4345 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.177738  4057 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.181026 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:03.178052  4345 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.183046  4115 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:03.184830 30882 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 15, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 15 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:03.186710 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:03.197561 31059 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:03.191591 31064 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:03.192874 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:03.213770 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.209008 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47203us
I20260709 12:26:03.098866  3581 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.187041  4057 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:03.194547  4317 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:26:03.252617 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 66020us
I20260709 12:26:03.189661  4283 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.133489  4321 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.112s
I20260709 12:26:03.259191 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.186918  4263 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:03.259896 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:03.267789 31698 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
W20260709 12:26:03.135380  4054 consensus_queue.cc:1175] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" responder_term: 15 status { last_received { term: 15 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 15 index: 13 } }
I20260709 12:26:03.206643  4176 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.264035  3893 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:03.280367 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 49085.1us
I20260709 12:26:03.295011 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.267408  3591 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:03.294503 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.298626 31138 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.299427 30984 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.307142 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.265532  4204 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:26:03.296933  4051 raft_consensus.cc:2764] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:03.280115  4032 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:26:03.307075  3893 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:03.331200  4263 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:03.323764  4290 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:03.347857 31697 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
W20260709 12:26:03.355538 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 96462.4us
I20260709 12:26:03.318436  3591 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.365725 31696 raft_consensus.cc:606] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.366199 31696 raft_consensus.cc:614] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:03.369576  3893 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.373308  4105 raft_consensus.cc:2764] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 12:26:03.376823 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.378746  4176 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:03.379488  3591 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:03.383752  4032 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:03.384869  4032 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:03.385231  4032 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:03.386925  3581 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.388756  4345 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:03.388971  3581 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:03.383986  4122 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:03.392901 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 132452us
I20260709 12:26:03.395274 30986 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.394943  4122 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.399194 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:03.411722 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.411722 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.414078 31691 raft_consensus.cc:2393] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 11: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:03.413909  4122 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:03.422236 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:03.422914 31694 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:26:03.426750 30990 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:03.427958 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.429054 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:03.429699 30990 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 12:26:03.429808 31141 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:03.430716 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.431500 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.432189 31692 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:03.432483 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:03.427977 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:03.416971 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:03.435413 31698 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:03.453493 31689 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:03.457381 30991 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:03.438282 31143 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:03.435566 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:03.441048 31063 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:03.462402 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:03.463519 31697 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:26:03.437539 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.462999 31142 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:03.465981 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.466639 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:03.466948 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:03.467118 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:03.355506  4263 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.467515 31696 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:26:03.467685 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.500211 31695 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.468367 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.468967  4115 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:03.466639 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.470396  4270 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:03.470598  4356 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:26:03.534343 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:03.471860 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.472894  4245 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:03.473557  4265 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:03.473811  4105 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:03.475708  4284 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:03.476815 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:03.479091 31141 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:03.478960 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:03.546274 31697 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:03.546168 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.479255 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:03.439846 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.482666 31138 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:03.386178  4032 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.498479 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.501780 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:03.521096 30991 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:03.532307 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:03.549011 30984 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:03.544222 31139 raft_consensus.cc:2368] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:03.414609  4057 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:03.548794 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.398406  4339 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.550271 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.531466  4360 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.556154 31698 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:03.528990  4270 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:03.559898 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.563155 31690 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:03.561293 31144 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:03.560379 30988 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:03.566537 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.568506 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.569105 31137 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:03.570662 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:03.571283 31690 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:03.571583 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.572091 31140 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.572171 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:03.573035 31064 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:03.577109 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:03.578488  4270 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:03.579635  4270 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:03.585408 31137 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
W20260709 12:26:03.551424 31033 connection.cc:256] RPC call timeout handler was delayed by -1.3351158309997118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
W20260709 12:26:03.586335 31033 connection.cc:256] RPC call timeout handler was delayed by -1.2577158309995866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
I20260709 12:26:03.587250  4225 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:03.588135 31135 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:03.591235 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.591884 31135 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:03.592484 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.592739 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.583752 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.593807 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.550314  4057 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:03.601737 31691 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:03.588584 30987 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:03.589161 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.589519 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.589946 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.596278 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:03.604660 31139 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:03.596472 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.593371  3581 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:26:03.599071 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.606751 31143 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:03.609360 31141 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:03.610198  4057 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.610960  4365 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:03.611382  4365 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:03.611666 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:03.612198  4365 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:03.611402 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.613629  4365 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.617491 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.618463 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.619040 31144 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.620029 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.620719 31139 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:03.621799 31139 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:03.623206 31141 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:03.624106 31143 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:03.618024 31697 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.607971  4051 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:26:03.634568  4242 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:03.635022  4242 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:03.635406 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.636163 30984 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:03.635888  4242 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:03.637300  4242 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:03.637629 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:03.639067 31691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45809554a739456ea0a596484ede5ac7"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:03.639612 31691 raft_consensus.cc:606] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.639966 31691 raft_consensus.cc:614] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:03.646637  4366 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:26:03.647864 31691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:03.652323 31691 raft_consensus.cc:606] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.654987 31691 raft_consensus.cc:614] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:03.606212  4307 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:03.665160 31059 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.689671 31695 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:03.693207 31693 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.678467 31689 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:26:03.689781 31140 raft_consensus.cc:2410] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:26:03.683792 31697 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:03.696473 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.697155 31139 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:03.697059 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:03.699505 31692 raft_consensus.cc:2410] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:03.701514 31064 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:03.701718 31691 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:03.692500 31144 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:03.705755 31694 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:03.706905 31068 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:03.722822 31697 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:26:03.724262 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 147234us
I20260709 12:26:03.726428 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.733479 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.735754 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.736691 30988 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:03.731761 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:03.564960  4360 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:03.723531 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.516s (SENT) [suppressed 33 similar messages]
I20260709 12:26:03.745529 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.699441  3893 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:03.740060 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:03.759076 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.759122 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:03.775941 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.776754 30988 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:03.780599 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.781575 30987 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 20: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:03.796253 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.799280 30988 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:03.809440 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:03.747982  3591 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:26:03.815007  4340 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:03.815383  4340 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:03.818926 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.815008 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.821626 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.827621 31138 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:03.830902  4339 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:03.831091 31141 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:03.834034 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.835765 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:03.839232 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59149.7us
I20260709 12:26:03.840041 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:03.845608 30986 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:03.846601 30987 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:03.846956 30987 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:03.847675 30987 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.768118  3893 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.851963  3893 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:03.852415  4054 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:03.854334 30990 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:03.857216 30988 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 13: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:03.859786 31059 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 12:26:03.861557 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.825009  4305 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:03.863967 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.866858 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.827075  4176 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:26:03.861593  4054 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:03.864995  4305 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:03.888051 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.870421  3581 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:03.889734 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:03.891191  3581 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:03.864724  4163 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:26:03.892957  4305 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:03.921895 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.896540  3591 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.928253  4054 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:03.965758 31059 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:03.965051  4051 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 12:26:03.971194  4371 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:26:03.971681  3591 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:03.991098 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:03.998438 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:04.002391 31068 raft_consensus.cc:2368] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16.
I20260709 12:26:04.014555 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:03.926942  3893 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:26:04.016607  3893 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:04.017037  3893 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.020545 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:03.993713  4262 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:26:04.037098  4321 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:04.037611  4321 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:04.032498  4360 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:04.043409 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.017973  3893 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.046057  3893 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.913358  3581 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.061913 31661 leader_election.cc:400] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16.
I20260709 12:26:04.062994 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.066847 31062 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:04.055584  4321 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.079329 30990 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:04.081494 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.084709 30986 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:04.088934  4198 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:04.089552  4082 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:04.092941  4307 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:04.138358  4082 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:04.138710  4307 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.095369 30987 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:04.098373  4032 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:04.102705  4368 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.368s	user 0.005s	sys 0.000s
I20260709 12:26:04.108474 31661 leader_election.cc:403] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:04.120987 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:04.139664 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 416132us
I20260709 12:26:04.146838 30984 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:26:04.147437 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 87382.2us
I20260709 12:26:04.158321  4027 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16.
I20260709 12:26:04.158445  4245 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.159011  4245 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.161067  4245 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:04.161343  4369 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 12:26:04.161497  4245 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.162701  4370 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 12:26:04.164158 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:04.167297 31140 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:04.167503  4270 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:04.168017  4270 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.121939  4198 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:04.161993  4245 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.169407  4198 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.170698  4198 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.171074  4245 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.172420 31693 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:26:04.173915 31697 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:04.175247 31692 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:04.176235 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:04.176867 31695 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:04.177358 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:04.177686  4345 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:04.178516  4265 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:04.139262  4082 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.179311  4082 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.146337 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.184293 30990 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:04.169979  4270 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:04.187145 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:04.189487 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:04.190049  4270 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:04.190347 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:04.190529  4270 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.191162  4265 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:04.191867  4270 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:04.192101  4340 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.193365  4345 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:04.193545  4340 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:04.196317  3591 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:04.196776  3591 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.197299  4245 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:04.189687 30989 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:04.141598  4365 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:26:04.199331 31690 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:04.201866  4307 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.202894  4307 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:04.203435  4270 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:04.204522 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 89793.6us
I20260709 12:26:04.197685  3591 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.207021 30987 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:26:04.210000 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.209679 30988 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:04.219033 30989 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: [15.7-16.8]
I20260709 12:26:04.212200  4321 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:04.242870  4305 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:04.244285 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.244732 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.245657  4305 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.250603  4057 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:04.258098 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.259531 30989 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:26:04.263739 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.267577 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 406368us
W20260709 12:26:04.274092 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 346367us
W20260709 12:26:04.232419 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 368540us
I20260709 12:26:04.277776 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:04.278584 31140 raft_consensus.cc:2368] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16.
I20260709 12:26:04.277776 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.280303 30989 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:04.281136 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:04.270449  4345 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:04.283492 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.271938  4290 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:26:04.295209 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.291090 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:04.273501  4305 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
W20260709 12:26:04.290350 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 99079.3us
I20260709 12:26:04.295271 31664 leader_election.cc:400] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16.
I20260709 12:26:04.287599  4265 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:04.292287  4225 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.289871  4359 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:04.315876 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.316440 30988 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:04.317414 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:04.308439  4225 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.318409 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:04.343941  4284 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:04.289261  4382 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:26:04.324074 30988 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:04.314960  4270 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:04.355086 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:04.355177 30989 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9.
I20260709 12:26:04.359901  4225 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:04.360255  4384 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:04.360770 30989 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:04.361608 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:04.361804 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:04.362573 30990 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:04.362581 30989 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 12: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:04.363662 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:04.364622 30984 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:04.365658  4225 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.365610 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:04.366436 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.366888  4225 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.308281  4032 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:26:04.367987  3591 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:04.369306  4225 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
W20260709 12:26:04.369879 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.370836 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:04.367237 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:04.371503 30987 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 11.
I20260709 12:26:04.371703 30988 raft_consensus.cc:2376] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 9. Re-sending same reply.
W20260709 12:26:04.373764 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 82134.3us
I20260709 12:26:04.373904  4242 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:26:04.374747 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 73109.8us
I20260709 12:26:04.374850 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:04.375547  4359 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.376518  4359 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.377302 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:04.378408  4263 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:04.378564  4365 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:04.379743  4377 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.131s	user 0.002s	sys 0.000s
I20260709 12:26:04.361562 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:04.378419 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:04.387773 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:04.386441 31143 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:26:04.392175 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:04.393303 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:04.398598 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:04.389894  4307 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:04.405516 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:04.405709 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:04.401391  4270 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:04.406687  4387 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.407101  4386 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:04.358121  4345 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.308642  4388 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.144s
I20260709 12:26:04.404014  4027 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:04.405129  4382 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:04.404980  3893 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:04.450374  4082 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:04.449628 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44198.4us
I20260709 12:26:04.463196 31059 raft_consensus.cc:2410] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 12:26:04.458349 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.464681 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59040.6us
I20260709 12:26:04.465065 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:04.412827  4307 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.452342  4027 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.452718  4245 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:04.478948 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:04.479939 31689 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:04.493845 31690 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:04.468055  4051 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:04.477993  4386 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:04.475281  4082 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:04.450623  4387 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.472543  3893 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.472824  4382 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.421506  4265 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.505554  4386 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.477496  4027 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:04.519894  4382 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:04.521477 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 48385.6us
I20260709 12:26:04.512177  4051 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.526994 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:04.528491 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:04.531234  4051 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.531144  4307 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.532896  4307 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:04.529909 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:04.544330 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73943.9us
I20260709 12:26:04.544553  3591 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:04.545676  3591 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:04.534102  3581 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:04.549961  3591 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.553828  4082 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:04.559902 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:04.568367  4395 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:04.569464  4057 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.570204  4057 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.569489  4395 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:04.572307  4395 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:04.566865  4388 consensus_queue.cc:1243] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:04.573335 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:04.574167 31143 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:04.575469 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:04.576100 31058 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:04.577100  4263 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:04.577554  4263 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:04.577936  4263 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:04.542598  4360 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:04.579562 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:04.587414 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:04.583287 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:04.598752 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:04.599498 31689 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:04.584033  4360 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:04.608153 31690 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:26:04.594964  4340 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:04.560951  4393 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 12:26:04.601437  4383 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:26:04.608640  3591 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:04.585144  4270 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:04.599855  4359 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:04.608310  4395 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.615306  4340 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.616859  3591 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.596578  4242 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:26:04.606878  4381 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.610934  4360 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:04.614130  4057 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.638669  4340 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:04.623998  4378 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.648988 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:04.652281  3591 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:04.652558 31143 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:04.642050  4284 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.643313  4359 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:26:04.664011 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.668457 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:04.669296  3591 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:04.647527  4395 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.602794  3893 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.672664  3591 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.682200  3591 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:04.645984  4381 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.682747  3591 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.668910  4386 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:04.685134  3591 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.695291 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:04.702265 31058 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:04.681564  4396 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:04.656302  4027 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.670961  4284 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.719424  3591 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:04.711546  4396 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.723905  4396 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.724574  4359 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.726867  4381 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:04.727963  4284 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.694638  4387 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.700104  4386 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:04.730764  4401 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:26:04.738384 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.741045  4386 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:04.657065  4378 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.758387 31143 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:04.760443 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:04.764237 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:04.765072  4386 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:04.768195 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:04.757908  4372 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:04.770699 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:04.771637 31137 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:04.769006 31141 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:04.777990 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:04.767956  4284 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.766610  4386 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:04.779943  4387 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:04.779323 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:04.798259  4378 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.800629  4390 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.801132  4390 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.803272  4390 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:04.807791  4027 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.787209  4396 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:04.808595  4396 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.808944  4396 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:04.809666 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:04.783695  4387 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:04.812427  4387 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:04.812755  4176 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:04.813257  4176 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:04.813551 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.814111  4176 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.783927  4284 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:04.833544 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 429165us
I20260709 12:26:04.886929 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:04.893409 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:04.900213  4245 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:04.900766  4245 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.907477  4396 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.910372  4396 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:04.912279  4245 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.914667  4245 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:04.927560 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:04.936506 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:04.938586 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:04.940621 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:04.965371 31067 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:04.968696 31067 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:04.970384 31067 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:04.999644  4390 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:05.001286  4390 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:05.002544  4390 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.011966  4345 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:05.012651  4345 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.015232  4345 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.025965  4390 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:05.027630  4390 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:05.028067  4390 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.028539  4390 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.031385 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.034077  4345 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:05.039467 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.009s	sys 0.005s
W20260709 12:26:05.039840 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.010s	sys 0.005s
I20260709 12:26:05.041929  4345 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:05.046259  4390 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.050125 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.050201  4390 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:05.051415  4390 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:05.051949 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.052225  4390 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:05.052721  4245 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.054595 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.053819  4390 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.055370  4245 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:05.055850  4245 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:05.056707  4245 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.051424  4345 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.067062 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.069486 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.070730  4270 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.070621 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.072060  4245 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.072211  4318 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:05.072919  4318 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:05.073366  4318 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:05.075017 30988 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:05.046207  4204 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:05.077576  4204 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:05.078306  4318 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.079411  4198 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:26:05.083020  4198 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:05.083369  4198 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:05.084139  4198 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.085330  4198 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:05.091037 30989 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:05.099428 30986 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:05.100358 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.101043 30989 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:26:05.110129 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.112010 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.112591 30990 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:05.113466 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:05.114068 30986 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.104979  4245 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:26:05.115805 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:05.117332  4262 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:05.117753  4262 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.118127  4262 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.119076 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.134080 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.135032 30986 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:05.136379 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:05.142182 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51800.8us
I20260709 12:26:05.143918  4339 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.144479  4339 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.146412  4305 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:05.130373  4204 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:05.154973  4263 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.570s	user 0.014s	sys 0.001s
I20260709 12:26:05.155262  4339 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:05.172892  4339 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:05.173426  4305 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:05.172591  4263 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.174589  4305 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.176160  4263 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:05.155794 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.163417 30987 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:26:05.157312 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.172575  4204 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.165900  4262 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.172150 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.186573  4290 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.187106  4290 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.187932  4204 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.189275  4290 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.191165  4345 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:05.191454  4290 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:05.192492  4204 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:05.193010  4204 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:05.194037  4204 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.195302  4204 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.183414  4318 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.193156  4290 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.191632  4345 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:05.222388 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 65139.7us
W20260709 12:26:05.232295 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.223623 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 67841.9us
I20260709 12:26:05.233132 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.233278  4290 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.234340  4245 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.233852 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.235599  4262 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.239398 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:05.223251  4318 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.243158  4318 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:05.243600  4318 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.244001  4318 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.245409 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.245827  4318 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.249076  4396 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.249614  4396 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.254359  4396 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:05.254676 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.250169 30987 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.260995  4345 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.247089 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.250607 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.269066 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.271540 30958 leader_election.cc:400] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14.
I20260709 12:26:05.272120 31062 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:05.272271 30986 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 20: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:05.273622 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.173418  4339 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.301327 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.328742 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:05.329443 31697 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.300905  4262 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:05.336876 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:05.342480 31692 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:05.337551 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:05.341315 30987 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:05.342971 31690 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:05.355233 31693 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:26:05.332847  4176 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:05.358753  4176 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:05.360281  4284 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.361040  4176 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.361621  4339 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:05.362982 31692 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:05.363492  4198 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.364540  4198 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.365150  4198 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:05.365229 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.365918 30986 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:05.330531  4290 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.366806  4290 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.369549 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:05.370426 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.373195 31697 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:05.363897 31690 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:05.378700 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:05.382375 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.361714  4176 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:05.337659  4318 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.404085  4318 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.404594  4318 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:04.830554  4386 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.378382  4290 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.391372  4396 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:26:05.407429  4318 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.368854  4345 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.379418 31692 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:05.366029  4198 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.393740  4270 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:26:05.346303  4262 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.403187  4176 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:05.409915  4245 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.410132  4345 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.584232  4176 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.473981  4396 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:05.472155  4204 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.435642  4395 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.592836  4204 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.616698 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 1509) took 1238 ms. Trace:
I20260709 12:26:05.617238 31059 rpcz_store.cc:276] 0709 12:26:04.377949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.387550 (+  9601us) service_pool.cc:224] Handling call
0709 12:26:05.616461 (+1228911us) spinlock_profiling.cc:243] Waited 74.4 ms on lock 0x7b54002832d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:05.616632 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171184384}
I20260709 12:26:05.600973  4395 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.624505  4388 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:05.627101 31067 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.625995  4198 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:05.436916 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.665864 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.673281  4388 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:05.655260  4360 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.008s	user 0.018s	sys 0.001s
I20260709 12:26:05.679075  4360 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.670698  4262 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:05.680604  4198 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.682961  4198 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.689954  4204 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:05.694571 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 393314us
I20260709 12:26:05.592358  4396 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.695331 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:05.707257 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 337760us
I20260709 12:26:05.709383 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:05.711138 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.694412  4305 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.719589 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.720372 30987 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:05.722263 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.724241 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.724560 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.725517 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.725752 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.736879 31661 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.738765 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.739063 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.739300 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.739392 31058 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:05.739794 31059 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:05.741173 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.741654  4386 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:05.742363  4386 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.743309 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.744185 31058 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:26:05.744683  4386 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.746254 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.746922 31059 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:26:05.749173 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.749421  4027 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.749630  4290 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:05.749814  4378 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:05.750365 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.750458  4378 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:05.750779 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.749964  4027 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.751439 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:05.751399  4378 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.752127 30992 raft_consensus.cc:2376] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17. Re-sending same reply.
I20260709 12:26:05.717803  4390 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:05.752732  4390 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.753041 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.753154  4027 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:05.753705  4027 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.748951  4305 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.754124 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.754967  4032 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.755504 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.755414  4032 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.755863  4305 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.756067 30991 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:05.757019 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.757352  4032 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.757819 30990 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:05.757222 30991 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:26:05.750113  4290 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.758972  4290 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:05.750488 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.760349  4339 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.760852 31661 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.761677 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.762372  4284 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:05.765091 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:05.751876 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.754474 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:05.706804  4387 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.894s	user 0.018s	sys 0.000s
I20260709 12:26:05.754047 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.766595 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.767200 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.763928 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.768148 30992 raft_consensus.cc:2376] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19. Re-sending same reply.
I20260709 12:26:05.769886 30991 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:26:05.771983 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:05.774686 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.775271 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.775524 30987 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:05.775705 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.776506 31058 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:05.778610  4262 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:05.779033  4262 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.754124  4027 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.779371  4262 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:05.781203  4027 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.781750  4027 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:05.783538  4396 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.710767  4204 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.784907  4435 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.785073  4339 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:05.785534  4339 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.785410  4435 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.785990  4339 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.787073  4360 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:05.788563  4395 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:05.789790 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.789867  4387 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:05.791026 31063 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:05.773743 31661 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:05.792804 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.775888 31059 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:05.796603 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:05.797319 30986 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:05.798537  4387 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.798952  4176 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:05.799683  4032 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.800098  4305 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:05.800526  4305 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.800859  4305 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.790707  4388 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:05.817008 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:05.818976 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.782130  4027 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.827168 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:05.827950 31059 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:05.830085 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.831766 30984 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:05.819360  4032 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:05.831974 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:05.834120 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:05.838604 31067 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:26:05.839031 31067 pending_rounds.cc:85] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 12
I20260709 12:26:05.839306 31067 pending_rounds.cc:107] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:26:05.839564 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:05.839701 30986 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:05.839588 31067 raft_consensus.cc:2889] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.853521 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1510) took 1466 ms. Trace:
I20260709 12:26:05.854053 31067 rpcz_store.cc:276] 0709 12:26:04.387384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.387532 (+   148us) service_pool.cc:224] Handling call
0709 12:26:04.400377 (+ 12845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.965258 (+564881us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400275220. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb80bdc0 00007f2eeb8961c0 00007f2eeb897231 00007f2eeb893d8f 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c
0709 12:26:05.625434 (+660176us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b58006944d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb7f00aa 00007f2eeb872447 00007f2eeb88afe7 00007f2eeb88c7de 00007f2eeb896b49 00007f2eeb897416 00007f2eeb893d8f 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4
0709 12:26:05.853216 (+227782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.853455 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833924224}
I20260709 12:26:05.826275  4027 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.835520  4032 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:05.842751  4388 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:05.850665  4387 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:05.884399  4032 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:05.890799 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080011d460 after lost signal to thread 29848
I20260709 12:26:05.888403  4395 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:05.892611  4395 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:05.862651  4321 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:26:05.894533  4387 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.894699  4395 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.895126  4387 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.894593 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.896113 30987 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:26:05.907714 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:05.908730 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:05.908933  4340 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.256s	user 0.005s	sys 0.012s
I20260709 12:26:05.909441 30987 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:05.909353  4340 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.909868  4032 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.910271 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.910920 30987 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:05.911706 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.911991  4032 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:05.912361 30987 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:05.913110 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:05.913770 30987 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:05.890867  4436 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:05.914001  4032 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:05.914427  4436 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.914810  4032 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.915674  4436 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:05.915797  4032 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.915014  4420 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.918886  4420 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.900553 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:05.920429  4388 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:05.923480 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.923534  4388 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:05.924705 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.925580  4386 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:05.925925  4386 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.926250  4386 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:05.927317  4057 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.021s	sys 0.033s
W20260709 12:26:05.927659  4057 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.145s	user 0.021s	sys 0.033s
I20260709 12:26:05.939827 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:05.940284 30990 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:26:05.945143 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.946487 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:05.966176 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 135600us
I20260709 12:26:05.930317  4284 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:05.969543 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:05.972281  4340 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:05.974576 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:05.975168 31059 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:05.954861  4396 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:05.982419 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:05.958746  4395 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:05.964627  4378 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:05.961787  4445 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:06.020839 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.021916 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:06.026166 31059 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:06.002239  4345 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:06.029796 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.027964 31067 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:06.008369  4378 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.043288 30989 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:06.063302  4447 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:06.064402  4388 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.064939  4388 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.066951  4388 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:06.028306  4393 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:06.008621  4284 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:06.034062  4445 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:06.070386  4445 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:06.071286  4445 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.033066  4378 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:06.080286 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.080909 31062 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:06.080636 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.083534 31067 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:06.086405  4345 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:06.085216 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.088795 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.100791 31058 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.090875  4032 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.099449  4290 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.110198  4450 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.114538  4290 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.114823  4450 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.091436  4262 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.117250  4450 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.117594  4262 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.099560  4027 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.119850  4027 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.121328 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.122962 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.124958 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.128612 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:06.130023 31013 debug-util.cc:398] Leaking SignalData structure 0x7b0800129860 after lost signal to thread 30924
I20260709 12:26:06.108850  4443 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:06.130913  4443 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.131224  4443 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:06.131356 31013 debug-util.cc:398] Leaking SignalData structure 0x7b0800103900 after lost signal to thread 31016
W20260709 12:26:06.131685 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080011dc80 after lost signal to thread 31017
I20260709 12:26:06.128660 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.133028 31065 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:06.090893  4339 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.108914  4393 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:06.135985 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:06.136179  4339 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:06.136605 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:06.137146 30988 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:06.137889 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:06.137825 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.138553 30987 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:06.140826 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:06.138602 31060 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:06.141170 31065 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:06.141896 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.146711 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.113230  4032 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:06.091264  4305 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.156579 30988 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:06.159014 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.160283 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.160802 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.160830 30987 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:06.161589 31065 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:06.163357 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:06.164228 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.143631  4395 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:06.166783 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:06.169543 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:06.170130 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.164816 31061 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:06.172633 31692 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:26:06.171842 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:06.165596  4057 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.176693  4057 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.178779  4057 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:06.180783 31060 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:06.182353  4345 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:06.189240  4387 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.160890  4270 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:06.191639  4436 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.193800  4436 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.167259  4393 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:06.175690 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.197568 31062 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:06.197829 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.202828 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.206945 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.201458  4447 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:06.217715 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.218932 31065 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:06.208289 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.195845  4450 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:06.195971  4032 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:06.234418 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.234756 31060 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:06.173589  4386 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.238548  4176 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:06.238942  4176 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.239161 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.240046  4424 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:06.240018 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.240566  4424 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.239324  4176 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.227650  4027 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:06.241074  4426 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:06.268683  4426 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.269073  4426 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.218027  4057 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:06.229113  4270 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:26:06.271638 31060 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:26:06.272051  4270 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:06.240849 31065 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:06.231595  4395 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:06.240955  4424 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.273082  4395 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.273597  4395 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:06.273911 31661 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:06.233129  4447 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.275317  4447 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.219400  4393 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:06.278901  4393 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:06.279873  4393 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.249989  4386 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:06.299667 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44548.8us
W20260709 12:26:06.315820 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 86862.4us
W20260709 12:26:06.319816 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 46009.3us
I20260709 12:26:06.322628 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:06.324311 31692 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:06.324605 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:06.325227 30989 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:06.324322 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:06.307356  4032 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.346146 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.347882 31065 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:06.321593  4434 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:06.317787  4450 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.355176  4450 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.356421  4390 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.321508  4421 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.350095  4340 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.320964  4057 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.336380  4457 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:06.319203  4395 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:06.380090 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080011d360 after lost signal to thread 31093
W20260709 12:26:06.380503 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080011d340 after lost signal to thread 31094
I20260709 12:26:06.344219  4027 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.383394 31065 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:06.386790 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.387358 31060 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:06.384689 31661 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:06.388269 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.396155 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:06.347906  4418 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:26:06.396991 31058 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:06.398898 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2690) took 1149 ms. Trace:
I20260709 12:26:06.286206  4426 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.377167  4305 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.399319 31068 rpcz_store.cc:276] 0709 12:26:05.248997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.385450 (+1136453us) service_pool.cc:224] Handling call
0709 12:26:06.398400 (+ 12950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.398580 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.398609 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.398647 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:06.398683 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.398734 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.398840 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:06.402774 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:06.403126 31065 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:06.366967  4443 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.395092  4463 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.416055 30989 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:06.381880  4457 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.397477  4428 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.375351  4340 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.401149  4396 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.425348 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:06.426810 31692 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.404868  4387 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:06.409663  4423 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:06.442449  4423 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:06.443019  4423 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:06.368523  4421 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.432367 31693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0afdb545bb8f485993259007a00c1841"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:06.432320  4057 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.389046  4032 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.433507  4284 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.434615  4459 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:26:06.435299  4340 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:06.438479 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 1.74333e+06us
W20260709 12:26:06.438887  4458 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:06.409965  4436 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:06.448096 31063 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:06.462239  4436 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.463486  4436 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.448609 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.425101  4463 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.465296  4447 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:06.465822  4447 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.466602 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.467065  4463 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.469089  4421 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.431234  4428 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.470781  4284 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.446422  4426 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.466778  4447 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.474635 31692 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:06.412487  4378 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.483490 31697 raft_consensus.cc:2368] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17.
I20260709 12:26:06.470939 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.423664  4429 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:26:06.485276 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:06.452275  4461 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:26:06.485931  4378 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.485327 31067 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:06.485868 31068 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:06.487383  4457 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.489131  4395 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.447324  4305 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.490470  4305 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.491211  4443 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.422538  4027 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.492569  4305 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.476159  4428 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.499442 31068 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:06.500950 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:06.501521 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:06.502328  4305 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:06.503683 31692 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:06.504308 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:06.505246  4387 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:06.506129 30985 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:06.485702  4345 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:06.507320 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:06.506500 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.511883 30989 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:06.513003 30991 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:06.516088  4284 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:06.518882 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.524353 30983 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:06.526572 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:06.527335 30991 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:06.527760 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.525730 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.531293 31061 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:06.535051 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:06.535972 30989 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:06.506902  4305 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:06.544193 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:06.545113 30989 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:06.545821 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:06.511137  4387 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:06.557454  4468 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:06.557905  4468 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.558238  4468 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:06.546869  4386 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:06.560307  4386 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.560642  4386 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.530680  4270 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:06.534932  4284 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:06.569802 31664 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.534533  4467 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 12:26:06.510346  4424 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.614393  4032 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:06.528595  4176 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.616366 30985 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
W20260709 12:26:06.617384 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 47661.2us
I20260709 12:26:06.621675 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:06.623263 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:06.626207 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 98541.2us
I20260709 12:26:06.627131 31691 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:06.607477  4340 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.630683 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:06.633877 31698 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.638878  4390 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:06.618330  4270 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:06.641346 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:06.650488 31689 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:06.628140  4450 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.670039 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 857) took 1342 ms. Trace:
I20260709 12:26:06.672899 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:06.673530 31698 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:06.628253  4032 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.677328 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.677770 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:06.692487 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.689996 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.702664 31068 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:06.703192 31058 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:06.673012 31695 rpcz_store.cc:276] 0709 12:26:05.327490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.327598 (+   108us) service_pool.cc:224] Handling call
0709 12:26:05.336821 (+  9223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.337036 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:05.711202 (+374166us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.859528 (+148326us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.523928 (+664400us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b18004805e0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:26:06.668697 (+144769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.669670 (+   973us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:06.669743 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.669832 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.669964 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1509,"prepare.queue_time_us":447743,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":249912,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":194,"spinlock_wait_cycles":24706944,"thread_start_us":655407,"threads_started":4,"wal-append.queue_time_us":346028}
I20260709 12:26:06.667780  4452 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:06.687388 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.680024  4428 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:06.714331 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:06.679775  4470 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:06.724752 31067 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.641222  4340 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.667502  4451 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:26:06.730283  4435 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.736388  4420 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:06.731061  4472 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:26:06.678277  4345 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:26:06.732616  4032 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.734455 31692 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:06.734828  4270 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:06.739918 31059 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:26:06.741348  4474 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:06.641633  4305 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.747118 31698 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:06.761562 31068 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:06.712888  4418 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:06.675827  4386 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:06.774721 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 62918.7us
I20260709 12:26:06.781263 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:06.788978 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 160726us
I20260709 12:26:06.790462 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.793081 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:06.678650  4450 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.736424  4176 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.652810  4387 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:06.825224 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 90227.4us
I20260709 12:26:06.685314  4468 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:06.833757 31067 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:06.845045 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.776487  4428 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:06.756546  4390 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:26:06.851740 31691 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:06.852419 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:06.825450  4421 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:06.861063 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 233903us
I20260709 12:26:06.857342 31694 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:26:06.858008 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.859889 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.862470 31058 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.860059 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:06.863646 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.825966  4032 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:06.864387 31059 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.861742 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:06.780426  4441 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 12:26:06.846172  4418 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:06.861490  4421 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.866127  4032 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.843034  4485 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:26:06.883854 31059 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:06.875406  4429 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:06.890650 30991 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:06.861701  4420 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:26:06.880170  4463 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:06.885339  4421 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.903674 30986 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:06.887315  4032 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:06.890358  4485 consensus_queue.cc:1243] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:06.913658 31058 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:06.892076  4429 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:06.918577  4423 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:06.919236  4476 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:06.909348 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:06.921232 30989 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:06.923125  4386 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.927300 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:06.927872 31064 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:06.931353 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:06.934123 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080011d260 after lost signal to thread 31109
W20260709 12:26:06.934823 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080021d140 after lost signal to thread 31138
I20260709 12:26:06.873909  4425 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 12:26:06.935115 31013 debug-util.cc:398] Leaking SignalData structure 0x7b0800306ca0 after lost signal to thread 31139
W20260709 12:26:06.947028 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080040efa0 after lost signal to thread 31168
W20260709 12:26:06.947379 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080038e840 after lost signal to thread 31169
W20260709 12:26:06.948801 31013 debug-util.cc:398] Leaking SignalData structure 0x7b08003bdea0 after lost signal to thread 31722
W20260709 12:26:06.949199 31013 debug-util.cc:398] Leaking SignalData structure 0x7b08003ad2e0 after lost signal to thread 3591
W20260709 12:26:06.949481 31013 debug-util.cc:398] Leaking SignalData structure 0x7b08002c29a0 after lost signal to thread 3893
W20260709 12:26:06.949796 31013 debug-util.cc:398] Leaking SignalData structure 0x7b08003fdc40 after lost signal to thread 4051
W20260709 12:26:06.950127 31013 debug-util.cc:398] Leaking SignalData structure 0x7b0800295f00 after lost signal to thread 4163
W20260709 12:26:06.950567 31013 debug-util.cc:398] Leaking SignalData structure 0x7b08004139e0 after lost signal to thread 4242
W20260709 12:26:06.951077 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080023c8e0 after lost signal to thread 4307
W20260709 12:26:06.951879 31013 debug-util.cc:398] Leaking SignalData structure 0x7b08002fee20 after lost signal to thread 4365
W20260709 12:26:06.952414 31013 debug-util.cc:398] Leaking SignalData structure 0x7b080033a560 after lost signal to thread 4381
W20260709 12:26:06.952752 31013 debug-util.cc:398] Leaking SignalData structure 0x7b0800407a60 after lost signal to thread 4382
I20260709 12:26:06.905666  4424 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:06.916430  4441 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:06.974664  4340 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:06.977470  4489 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:06.927294  4450 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:07.001924 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2614) took 2205 ms. Trace:
I20260709 12:26:06.981843  4441 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:07.003757 31139 rpcz_store.cc:276] 0709 12:26:04.792128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.792358 (+   230us) service_pool.cc:224] Handling call
0709 12:26:04.792750 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.792971 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:04.793313 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.996292 (+2202979us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.996365 (+    73us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:06.996520 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.996938 (+   418us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.997061 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2177882,"prepare.queue_time_us":3775,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":2185130,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":24752,"threads_started":1}
I20260709 12:26:06.996646  4176 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:07.015776  4486 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:07.018105  4307 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:07.018814  4307 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.021152  4307 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:07.022044  4360 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.022241  4359 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:07.022609  4360 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.022969  4359 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.024673  4360 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:07.025156  4360 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.025390  4452 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.025596  4359 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:07.026167 31140 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:07.027577 31136 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:07.028203 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:07.028702 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.029244 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.030464 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.030898 31106 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:07.033597  4507 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.034039  4507 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.036005  4507 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:07.036336  3591 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.036438  4507 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.037333 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.037490  4051 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.037955  4051 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.039929  4051 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:07.040390  4051 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.028767 31137 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:26:07.036810  3591 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.031026 31135 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:07.025801  4452 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:06.904013  4428 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.047032 31064 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:07.049656 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:07.049824 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45563.8us
I20260709 12:26:07.050429 31059 raft_consensus.cc:2368] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:26:07.021744  4307 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.052754 30991 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:07.056433 31067 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:07.011432  4393 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.054036 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.057294 31064 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:07.061220 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.061866 31144 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:07.063199 31058 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:07.057760 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.040792  4051 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.059947 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:07.066045  3591 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:07.053396 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:07.067624 31139 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:07.067833 31141 raft_consensus.cc:2410] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:07.042273  4176 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:07.070338 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:07.053017  4263 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:07.127959 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:06.999150  4491 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 12:26:07.131587 31058 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:07.138299 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:07.137471 31064 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:07.140021 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:07.131574 30991 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:07.133327  3591 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:07.134210 30986 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:07.053092  4305 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:07.142197 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.066733  4463 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:07.142145  3591 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.036890  4507 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.025599  4360 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.158565 30991 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:07.159610 31106 leader_election.cc:400] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:26:07.163628 31106 leader_election.cc:403] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:07.136411  4263 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.159972 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:07.141850  4472 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:07.159720 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:06.994266  4418 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.143355  4381 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.202167 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2697) took 1111 ms. Trace:
I20260709 12:26:07.202306  4381 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.202602 31062 rpcz_store.cc:276] 0709 12:26:06.091173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.091754 (+   581us) service_pool.cc:224] Handling call
0709 12:26:06.092094 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.197937 (+105843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.201944 (+1004007us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54002832d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:07.202111 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237608192}
I20260709 12:26:07.143113  4382 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.068423  4307 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.168047 31061 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:07.197953 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:07.199024  4431 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:26:07.205076 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:07.200254  4498 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:26:07.200932  4507 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:07.170437  4263 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.218639  4506 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.219128  4506 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.221108  4506 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:07.221544  4506 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:07.222628  4508 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.223165  4508 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.225288  4508 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:07.228268  4419 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:07.228797  4419 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:07.208264 31139 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:07.201270  4429 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.232967  4429 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:07.234915  4505 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:26:07.235558  4263 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:07.212987  4502 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:26:07.213335  4381 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:07.213893  4382 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.216084 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.247642 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:07.216281  4307 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:07.216763  4501 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:07.074131  4393 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.240118  3591 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:07.210956 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:07.244457 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:07.245723 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 90407.9us
I20260709 12:26:07.248226 30984 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:07.221973  4506 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.257591 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:07.259306 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48432.4us
I20260709 12:26:07.158286  4305 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.260978 31136 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:07.268584 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:07.273834 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.240717  4508 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:07.274972 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.276244 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.276827 31136 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:07.252954  4507 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:07.278059 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.279637 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.280198 31141 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.283775  3591 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:07.283825 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:07.252838  4424 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.287637 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 240803us
I20260709 12:26:07.290113 31693 raft_consensus.cc:606] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:07.290902 31136 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:07.292618 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:07.292909 31690 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:07.293732  3591 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:07.294106  3591 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:07.295801 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.295884 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.266664  4420 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:07.229743  4419 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.293236 30991 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:07.297701 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.298908 31033 leader_election.cc:400] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17.
I20260709 12:26:07.294795  3591 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.307402 31140 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:07.272719  4501 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:07.269264  4307 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:26:07.168879  4441 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.196801  4176 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.273867  4463 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.344568 31063 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:07.823869 31033 leader_election.cc:403] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:07.823117 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.824463  3591 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:07.457022 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 258541us
I20260709 12:26:07.825479 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:07.827656 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.344858 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.830962 31137 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:07.321945 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:07.834029 31136 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:07.524503 31141 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:07.837677 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.843451 31141 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:07.308036 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.844659 31144 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:07.434113  4263 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:07.848074 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:07.894531 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.895834 31691 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:07.897215 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.897907 31137 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
W20260709 12:26:07.818753  4513 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
W20260709 12:26:07.819625  4503 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 12:26:07.905094 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.905369 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.905478 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:07.296949  4443 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:07.433684  4507 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:26:07.720635  4393 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:07.308408  4390 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:07.846961  4515 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:07.911098 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.823077  4491 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.905709 31141 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:07.918638 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.307585  3893 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:07.913076 31691 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:26:07.942377 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:07.295341  4508 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.944494 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:07.297701  4032 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.980842 31137 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:07.897753  4445 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17.
I20260709 12:26:07.306030  4424 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.989574 31691 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:07.911182  4419 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:08.047076 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:08.048192 31137 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:07.396044  4359 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.524s
I20260709 12:26:08.041173 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:08.067225 31140 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:08.043130 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:07.929090  4515 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.049212 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:08.068540 31137 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:08.047365 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:07.651721 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:08.063426 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:08.064476 31141 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:07.929266  4443 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:08.136178 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:07.720431  4051 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:08.116163 30986 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:08.170181 31691 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:07.990100  4382 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:08.150141 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.004s	user 0.002s	sys 0.007s
W20260709 12:26:08.175501 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.030s	user 0.002s	sys 0.007s
W20260709 12:26:08.176009 31109 connection.cc:256] RPC call timeout handler was delayed by -1.1470967320001364s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 12:26:08.040496  4360 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:08.154703 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.170459 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.171049 31137 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:26:08.181169 31141 raft_consensus.cc:2410] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 12:26:07.513099 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.078634  4513 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:26:08.183565 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2713) took 1496 ms. Trace:
W20260709 12:26:07.428041  4502 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:26:08.179306 31144 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:08.179889 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:08.182142 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.985304  3893 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:08.078331  4509 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.106169  4515 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:08.190442 31060 rpcz_store.cc:276] 0709 12:26:06.687615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.687870 (+   255us) service_pool.cc:224] Handling call
0709 12:26:06.688156 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.688464 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.688503 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.688951 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.061174 (+372223us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:07.061224 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.061286 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.183274 (+1121988us) raft_consensus.cc:1685] finished
0709 12:26:08.183305 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.183489 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16250,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":8487,"thread_start_us":371841,"threads_started":1,"wal-append.queue_time_us":598029}
I20260709 12:26:08.197765 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.198336 31060 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:07.983846  4491 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:07.415048  4426 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:08.199590  4426 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:08.200836  4491 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:08.200456  4426 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.201954  4426 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:08.203110  4512 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.513s	user 0.001s	sys 0.000s
W20260709 12:26:08.204734  4519 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:26:08.176504 31109 connection.cc:256] RPC call timeout handler was delayed by -1.033696731999953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
I20260709 12:26:08.174012 31058 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:08.196496 31137 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:08.206053 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:08.208577 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:08.209386 31063 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:07.395696  4380 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:08.211390  4380 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.213471  4380 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:08.214565  4380 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:08.214932  4382 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:08.215806  4382 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:08.216952  4426 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:08.217545  4426 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.219679 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2612) took 3458 ms. Trace:
I20260709 12:26:08.175122  4359 consensus_queue.cc:1243] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 15.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:08.206032  4515 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:07.347399  4420 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:08.220773 31138 rpcz_store.cc:276] 0709 12:26:04.760885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.761107 (+   222us) service_pool.cc:224] Handling call
0709 12:26:04.761425 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.143708 (+2382283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.143758 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.155588 (+ 11830us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.306856 (+151268us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:07.306892 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.306959 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.219410 (+912451us) raft_consensus.cc:1685] finished
0709 12:26:08.219442 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.219614 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2198133,"raft.queue_time_us":2216719,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":291048,"threads_started":2,"wal-append.queue_time_us":1008933}
I20260709 12:26:08.233585 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.218338  4509 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.235845 31064 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:08.237457 31058 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:08.238412 31060 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:08.239250 31063 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:26:08.245780  4452 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.190s	user 0.006s	sys 0.000s
I20260709 12:26:08.247994 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:07.424786  4511 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.968s
I20260709 12:26:08.286304 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.251492  4468 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:08.252640  4421 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.316s	user 0.004s	sys 0.006s
I20260709 12:26:08.254025 30991 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:08.264515 31142 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:08.291886 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:08.292583 31142 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:08.294517 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.270045  4506 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:08.300720 30989 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.010s	sys 0.012s
I20260709 12:26:08.224964  4441 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:08.301064 30989 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.010s	sys 0.012s
I20260709 12:26:08.267316 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.195569  4305 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:08.177834  4501 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:08.185123  4445 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:08.272848  4429 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.039s	user 0.011s	sys 0.000s
I20260709 12:26:08.273470 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59170 (request call id 19) took 1842 ms. Trace:
I20260709 12:26:08.309031 31693 rpcz_store.cc:276] 0709 12:26:06.431237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.431392 (+   155us) service_pool.cc:224] Handling call
0709 12:26:08.273286 (+1841894us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b5400690354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88e853 00007f2ef085c60a 00007f2ee9de27d4 00007f2ee9de2762 00007f2ee9de2691 00007f2ee9de260c 00007f2ee9de18c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:08.273378 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1938305792}
I20260709 12:26:08.274312  4423 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:08.277367  4504 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.047s	user 0.015s	sys 0.047s
I20260709 12:26:08.286664 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1236) took 1662 ms. Trace:
I20260709 12:26:08.284233 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:08.288196 31063 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:08.288098 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.317370 31060 raft_consensus.cc:2410] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:08.285720  4490 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:08.265282 31140 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:08.153033 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.215929  4380 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:08.304716 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 873) took 1417 ms. Trace:
I20260709 12:26:08.329525 31698 rpcz_store.cc:276] 0709 12:26:06.887786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.891498 (+  3712us) service_pool.cc:224] Handling call
0709 12:26:06.906242 (+ 14744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.916796 (+ 10554us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:06.962286 (+ 45490us) log.cc:844] Serialized 14 byte log entry
0709 12:26:07.157225 (+194939us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.157619 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.157760 (+   141us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:07.157789 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.157850 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.304511 (+1146661us) raft_consensus.cc:1685] finished
0709 12:26:08.304532 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.304660 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80870,"prepare.queue_time_us":45273,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":4719,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":7357,"thread_start_us":145868,"threads_started":1,"wal-append.queue_time_us":302040}
I20260709 12:26:08.287365  4359 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:26:08.332118 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.332855 31140 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
W20260709 12:26:08.321570 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 1.02833e+06us
I20260709 12:26:08.325377 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.327771 30986 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:26:08.336287 30986 pending_rounds.cc:85] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 11
I20260709 12:26:08.336602 30986 pending_rounds.cc:107] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:26:08.337679 30986 raft_consensus.cc:2889] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:08.290548  4452 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.299271  4421 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:08.331766  4380 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:08.315963  4429 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.313810 31696 rpcz_store.cc:276] 0709 12:26:06.624693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.627533 (+  2840us) service_pool.cc:224] Handling call
0709 12:26:06.627897 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.844719 (+216822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.844748 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.850932 (+  6184us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.885643 (+ 34711us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:06.885998 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.968426 (+ 82428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.968533 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.286453 (+1317920us) raft_consensus.cc:1685] finished
0709 12:26:08.286479 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.286608 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63755,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":2308,"raft.queue_time_us":132677,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":18993,"spinlock_wait_cycles":187648,"thread_start_us":225799,"threads_started":2,"wal-append.queue_time_us":135452}
W20260709 12:26:08.378448 31143 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.093s	sys 0.198s
I20260709 12:26:08.374115 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:08.380316 31698 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:08.376343 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:08.346858 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.381353 31144 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
W20260709 12:26:08.382025 31143 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.099s	user 0.093s	sys 0.199s
I20260709 12:26:08.382992 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:08.339553  4506 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:08.393210 31698 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:26:08.386924 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2460) took 1347 ms. Trace:
I20260709 12:26:08.388489  4424 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:08.332733  4490 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.378324  4380 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.314910  4445 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.389225 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.389973 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1138) took 1551 ms. Trace:
I20260709 12:26:08.391726  4506 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.388345 30984 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:08.299896  4443 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:08.395262 31143 rpcz_store.cc:276] 0709 12:26:07.039156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.039868 (+   712us) service_pool.cc:224] Handling call
0709 12:26:07.040181 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.040535 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.040566 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.040999 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.264618 (+223619us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:08.382410 (+1117792us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.386638 (+  4228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.386713 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.386750 (+    37us) raft_consensus.cc:1685] finished
0709 12:26:08.386761 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.386871 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4679,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":8439,"thread_start_us":1322249,"threads_started":2,"wal-append.queue_time_us":372817}
I20260709 12:26:08.388044 31142 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:08.429126 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:08.429314 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:08.429756 31696 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.429813 31142 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 17
W20260709 12:26:08.417582 31033 connection.cc:256] RPC call timeout handler was delayed by -1.1135698590001084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
I20260709 12:26:08.388381  4051 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:08.378562  4491 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:08.390098 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:08.426050 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.129s	user 0.026s	sys 0.070s
I20260709 12:26:08.335476  4420 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:08.417124  4468 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.432320 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.135s	user 0.026s	sys 0.070s
I20260709 12:26:08.425498 30983 rpcz_store.cc:276] 0709 12:26:06.838532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.838700 (+   168us) service_pool.cc:224] Handling call
0709 12:26:06.839053 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.849175 (+ 10122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.849218 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.857552 (+  8334us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.004916 (+147364us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:07.004984 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.005949 (+   965us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.276798 (+1270849us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:08.389661 (+112863us) raft_consensus.cc:1685] finished
0709 12:26:08.389686 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.389896 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12766,"raft.queue_time_us":14518,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7371,"spinlock_wait_cycles":379774848,"thread_start_us":126323,"threads_started":1,"wal-append.queue_time_us":253214}
I20260709 12:26:08.435587  4468 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.436275 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.436529  4490 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:08.436898 31140 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:26:08.437774 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.435608  4380 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:08.436209 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 1.14878e+06us
I20260709 12:26:08.440018 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:08.440945  4424 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:08.441434  4424 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:08.442276  4424 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:08.445624 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.446391 31061 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:08.448519 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:08.449220 31061 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
W20260709 12:26:08.450949 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.451458 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.455480  4494 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.522s	user 0.000s	sys 0.009s
I20260709 12:26:08.459833 31142 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:08.461414 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.462014 31142 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:08.464016 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.488751 31140 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:08.528268 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:08.662549 31142 raft_consensus.cc:2410] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 21 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:08.663640 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:08.665019 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:08.431105  4506 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.434157  4032 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:08.620867 30985 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:26:08.463150  4491 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:08.634524 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 423657us
I20260709 12:26:08.592773  4426 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:08.629877 31663 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.007s	sys 0.019s
I20260709 12:26:08.672884 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:08.996340 31663 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.141s	user 0.007s	sys 0.019s
I20260709 12:26:08.634375  4509 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.016551 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:09.017306 31142 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:08.385488  4429 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:08.635687  4452 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:08.982280 30957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.008s	sys 0.040s
W20260709 12:26:09.101940 30957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.008s	sys 0.040s
I20260709 12:26:08.996387 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:09.012205  4426 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:09.039589 30958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.008s	sys 0.046s
I20260709 12:26:09.103390 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:09.113662 30958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.009s	sys 0.047s
I20260709 12:26:09.113814 31143 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:09.121737  4032 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:09.131242  4441 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.823s	user 0.004s	sys 0.005s
I20260709 12:26:09.144908 31142 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
W20260709 12:26:09.104462  4423 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.009s	sys 0.032s
I20260709 12:26:09.160645 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:09.161468 30989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.004s	sys 0.033s
I20260709 12:26:09.134883  4421 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:09.150199  4463 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.022s	sys 0.016s
W20260709 12:26:09.181551 31661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.011s	sys 0.027s
I20260709 12:26:09.159480  4419 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:09.189527 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.033s	sys 0.072s
W20260709 12:26:09.189843 31061 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.012s	sys 0.048s
W20260709 12:26:09.159595  4418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.003s	sys 0.034s
W20260709 12:26:09.166679  4390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.119s	user 0.020s	sys 0.048s
I20260709 12:26:09.138092  4426 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:09.198211 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.036s	sys 0.023s
I20260709 12:26:09.201200 31143 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
W20260709 12:26:09.302505 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.036s	sys 0.023s
W20260709 12:26:09.182531  4420 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.028s	sys 0.029s
I20260709 12:26:09.137948  4032 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.203804 31144 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:09.205540 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.327746 31142 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
W20260709 12:26:09.215785 30989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.004s	sys 0.033s
W20260709 12:26:09.191285  4424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.013s	sys 0.049s
W20260709 12:26:09.217410 31661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.011s	sys 0.027s
W20260709 12:26:09.218138 31032 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.015s	sys 0.018s
W20260709 12:26:09.373771 31032 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.015s	sys 0.018s
W20260709 12:26:09.236553  4390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.195s	user 0.021s	sys 0.048s
W20260709 12:26:09.235984 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.038s	sys 0.072s
W20260709 12:26:09.237963 31061 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.016s	sys 0.048s
I20260709 12:26:09.301532 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1245) took 2582 ms. Trace:
I20260709 12:26:09.379940 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:09.380612 31142 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:09.222781 31030 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.003s	sys 0.016s
W20260709 12:26:09.217991  4388 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.007s	sys 0.007s
W20260709 12:26:09.302899 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 843321us
I20260709 12:26:09.192719  4527 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:09.303784 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:09.214694  4423 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.009s	sys 0.032s
W20260709 12:26:09.328869  4420 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.029s	sys 0.030s
I20260709 12:26:09.392591 31140 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 13: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:09.335848 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1140) took 1639 ms. Trace:
I20260709 12:26:09.222085 31059 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:09.394294 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2947) took 2073 ms. Trace:
W20260709 12:26:09.383478  4388 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.007s	sys 0.008s
W20260709 12:26:09.410444 31696 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.980s	user 0.010s	sys 0.000s
I20260709 12:26:09.410880 31696 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:09.411474  4388 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:09.412164  4388 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.393258  4527 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.425575  4527 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.386286  4452 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:26:09.428117  4452 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:09.429605  4491 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.431836  4524 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.432405  4524 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.434716  4429 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.435339  4429 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.435856  4429 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.327421  4421 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:09.437711  4421 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.440164  4421 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.443709  4426 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.325817 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1083) took 1614 ms. Trace:
W20260709 12:26:09.373384  4424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.014s	sys 0.049s
W20260709 12:26:09.219295  4463 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.022s	sys 0.016s
W20260709 12:26:09.302675  4418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.004s	sys 0.037s
W20260709 12:26:09.381641 31030 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.005s	sys 0.018s
I20260709 12:26:09.380076 31690 rpcz_store.cc:276] 0709 12:26:06.637931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.730544 (+ 92613us) service_pool.cc:224] Handling call
0709 12:26:06.730835 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.293405 (+562570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.220547 (+1927142us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b5400690354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:09.220714 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292290432}
W20260709 12:26:09.384546 31031 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.000s	sys 0.025s
W20260709 12:26:09.473544 31031 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.000s	sys 0.025s
I20260709 12:26:09.383921 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:09.219612  4521 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:09.326829  4522 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.394646 31059 rpcz_store.cc:276] 0709 12:26:07.321007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.396827 (+ 75820us) service_pool.cc:224] Handling call
0709 12:26:09.221810 (+1824983us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b5800129398. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:26:09.221955 (+   145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.394019 (+172064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.394236 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13100928}
I20260709 12:26:09.397636 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 1094) took 1017 ms. Trace:
I20260709 12:26:09.482936 31144 rpcz_store.cc:276] 0709 12:26:08.380518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.382032 (+  1514us) service_pool.cc:224] Handling call
0709 12:26:09.397417 (+1015385us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b54003f4dd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:09.397565 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790329728}
I20260709 12:26:09.376411 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.398849 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:09.484580 31140 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.400102  4360 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:26:09.400548  4416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.933s	user 0.039s	sys 0.059s
I20260709 12:26:09.400983  4529 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:09.406610  4525 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:09.407709  4263 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:09.408681 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.503962 31138 raft_consensus.cc:2410] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:09.384996  4390 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:09.446797 31136 rpcz_store.cc:276] 0709 12:26:07.711678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.989648 (+277970us) service_pool.cc:224] Handling call
0709 12:26:08.040851 (+ 51203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.079623 (+ 38772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.079667 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.079720 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:08.079774 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.325557 (+1245783us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b54001ea0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:09.325615 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.325749 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88642944}
I20260709 12:26:09.448484  4491 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:09.458511 30989 rpcz_store.cc:276] 0709 12:26:07.699686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.702619 (+  2933us) service_pool.cc:224] Handling call
0709 12:26:07.703010 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.703253 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:08.308397 (+605144us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.335409 (+1027012us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.335477 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:09.335552 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.335641 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.335783 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":928233,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":106451,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":1566899,"threads_started":2}
I20260709 12:26:09.508965 31136 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:09.509318 31136 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:09.511039 31136 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.521241 31136 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:09.508147  4416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.305s	user 0.039s	sys 0.060s
I20260709 12:26:09.492166  4360 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:09.301332  4419 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:09.506441 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 122604us
W20260709 12:26:09.451153 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 469302us
W20260709 12:26:09.531685 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 373332us
I20260709 12:26:09.510635  4424 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:09.507486  4263 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:09.533784 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:09.535367 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0328253120001136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 12:26:09.535789 31033 connection.cc:256] RPC call timeout handler was delayed by -1.0272175000000061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 12:26:09.539680 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.540798 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.544097 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:09.512869 31144 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:09.545601 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2637) took 1174 ms. Trace:
I20260709 12:26:09.546020 31144 rpcz_store.cc:276] 0709 12:26:08.370964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.497751 (+1126787us) service_pool.cc:224] Handling call
0709 12:26:09.512557 (+ 14806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.545264 (+ 32707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.545516 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.471438 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2466) took 2218 ms. Trace:
I20260709 12:26:09.551961 31139 rpcz_store.cc:276] 0709 12:26:07.253215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.253985 (+   770us) service_pool.cc:224] Handling call
0709 12:26:07.269701 (+ 15716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.296372 (+ 26671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.296402 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:07.296442 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:07.296479 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.471150 (+2174671us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b540001bb54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:09.471211 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.471367 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60927360}
I20260709 12:26:09.530040 31140 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:09.506923  4463 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.533505 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:09.496656  4522 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.555603  4426 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.496618  4521 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.548338  4418 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.559026 31140 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:09.506305  4529 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.537020  4424 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.512348  4491 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.554904  4345 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:09.548775  4544 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:09.567909 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.553864 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1238) took 2924 ms. Trace:
I20260709 12:26:09.567929 30987 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:26:09.576417 30987 pending_rounds.cc:85] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 9
I20260709 12:26:09.576717 30987 pending_rounds.cc:107] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:26:09.577107 30987 raft_consensus.cc:2889] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.548853  4541 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.548921  4524 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.564134  4420 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:09.588452  4538 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:26:09.591230 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1147) took 1284 ms. Trace:
I20260709 12:26:09.591639 30986 rpcz_store.cc:276] 0709 12:26:08.307101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.308090 (+   989us) service_pool.cc:224] Handling call
0709 12:26:08.308417 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.338252 (+ 29835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.590999 (+1252747us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b54003467d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:09.591169 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43579136}
I20260709 12:26:09.568950 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:09.527859 31108 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.012s	sys 0.006s
W20260709 12:26:09.593740 31108 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.012s	sys 0.006s
I20260709 12:26:09.594974 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1144) took 1314 ms. Trace:
W20260709 12:26:09.573347 30958 connection.cc:256] RPC call timeout handler was delayed by -1.2977300699999432s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
I20260709 12:26:09.576331 31694 rpcz_store.cc:276] 0709 12:26:06.629077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.632152 (+  3075us) service_pool.cc:224] Handling call
0709 12:26:06.632496 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.861665 (+229169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.548672 (+2687007us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540011b7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:09.553807 (+  5135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516129664}
I20260709 12:26:09.579725 30987 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:26:09.602178 30958 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.539s (SENT) [suppressed 29 similar messages]
I20260709 12:26:09.566155  4545 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:09.603960 31140 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:09.604475 31140 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:09.574841  4424 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.580402 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:09.581056 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:09.614830 30989 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.558029  4388 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.488232 31142 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:09.564492  4429 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.579378 31144 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:09.607352 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2797) took 2277 ms. Trace:
I20260709 12:26:09.578728  4544 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:09.564020  4463 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.611819 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.612600  4423 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.612854 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.595435 30985 rpcz_store.cc:276] 0709 12:26:08.280999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.281370 (+   371us) service_pool.cc:224] Handling call
0709 12:26:08.286071 (+  4701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.786095 (+500024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.594698 (+808603us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54001608d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:09.594903 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733678848}
I20260709 12:26:09.644799 31063 raft_consensus.cc:2410] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:09.609503 31136 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:26:09.617960 30984 rpcz_store.cc:276] 0709 12:26:07.308209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.308576 (+   367us) service_pool.cc:224] Handling call
0709 12:26:07.308905 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.427542 (+1118637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.585331 (+1157789us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540029a9d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:09.585530 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2482054912}
I20260709 12:26:09.646106 31136 pending_rounds.cc:85] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 7
I20260709 12:26:09.646654 31136 pending_rounds.cc:107] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 17.8
I20260709 12:26:09.647109 31136 raft_consensus.cc:2889] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:09.647737 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 261914us
I20260709 12:26:09.648025 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2639) took 1273 ms. Trace:
I20260709 12:26:09.648644 31136 rpcz_store.cc:276] 0709 12:26:08.375046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.506707 (+1131661us) service_pool.cc:224] Handling call
0709 12:26:09.508840 (+  2133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.647738 (+138898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.647967 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.650002 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:09.567144  4426 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.654930 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:09.642556 31140 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.664798 31060 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:09.665237 31140 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.670605 31063 raft_consensus.cc:1217] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Deduplicated request from leader. Original: 14.12->[16.13-20.15]   Dedup: 16.13->[18.14-20.15]
I20260709 12:26:09.671093 31063 pending_rounds.cc:85] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 13
I20260709 12:26:09.671325 31063 pending_rounds.cc:107] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:26:09.671564 31063 raft_consensus.cc:2889] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.660538 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:09.662895 31065 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:09.582543  4345 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.617555  4524 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.687161 31059 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.694831 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2727) took 1242 ms. Trace:
I20260709 12:26:09.695216 31061 rpcz_store.cc:276] 0709 12:26:08.452905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.453069 (+   164us) service_pool.cc:224] Handling call
0709 12:26:08.453380 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.453611 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:09.379751 (+926140us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.694512 (+314761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.694572 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:09.694613 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.694674 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.694781 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36088,"prepare.queue_time_us":1037652,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":1204042,"threads_started":2}
I20260709 12:26:09.621853  4535 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.696934 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.692670 30989 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
W20260709 12:26:09.697947 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 95785.7us
I20260709 12:26:09.704842 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.706792 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:09.707340 30958 connection.cc:256] RPC call timeout handler was delayed by -1.2005743750000875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
I20260709 12:26:09.677078  4545 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:09.700889 31060 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:09.577914  4360 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.574259  4418 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.567284  4390 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.622274  4540 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:09.685599  4380 consensus_meta.cc:287] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.247s	user 0.011s	sys 0.000s
I20260709 12:26:09.585361  4541 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.700429 31140 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:26:09.736434 31140 pending_rounds.cc:85] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 12
I20260709 12:26:09.736706 31140 pending_rounds.cc:107] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:26:09.736981 31140 raft_consensus.cc:2889] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.718014 31059 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:09.676640  4032 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:09.662500  4544 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:09.675465  4509 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:09.680979  4554 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:09.729499 30958 connection.cc:256] RPC call timeout handler was delayed by -1.186283689999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21240000000000003s for a response
I20260709 12:26:09.735458 31694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "205fb3f3a3844073a49da13542da2989"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:09.737782 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1113) took 1088 ms. Trace:
I20260709 12:26:09.746809 31694 raft_consensus.cc:606] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:09.738873 30990 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:09.747224 31694 raft_consensus.cc:614] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:09.739408 31143 raft_consensus.cc:2410] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:09.680083  4057 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:26:09.676960  4525 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
W20260709 12:26:09.741076 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 171694us
I20260709 12:26:09.723275  4545 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.679745  4423 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.748245 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.759406  4345 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.746743 31140 rpcz_store.cc:276] 0709 12:26:08.649459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.579466 (+930007us) service_pool.cc:224] Handling call
0709 12:26:09.603817 (+ 24351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.737516 (+133699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.737722 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.760769  4521 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.762760  4443 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.741672  4380 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.767153 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:09.708425  4524 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.739431  4543 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.728659  4529 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.741847  4552 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.804674  4549 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:09.805157  4549 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.804939  4552 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.805557  4549 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:09.774205 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.735911  4540 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.807209 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.807797 31140 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:09.808652 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.808892  4540 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.774562  4382 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:26:09.778867 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.781327  4345 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:09.809813  4382 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:09.810209  4382 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:09.809981  4345 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.810385 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.810976 31065 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:09.811041  4382 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.811851 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.812073  4345 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.812258  4382 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:09.776683  4548 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:26:09.812611 31065 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:09.786028  4443 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.813114  4543 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.773192  4545 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:09.815325  4543 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.711515  4535 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.818675 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:09.772682  4418 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.823683 31064 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:09.761554  4419 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.824635  4535 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.825042 31061 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:09.793853 31694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48047b2b139146c38a821328cf7bd148"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:09.829209 31694 raft_consensus.cc:606] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:09.831626  4463 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.832211  4515 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:09.833155 31141 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:09.833496 31141 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:09.833724 31061 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:09.834185 31141 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.835146 31141 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:09.836194  4506 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.821s	user 0.009s	sys 0.000s
I20260709 12:26:09.836589  4506 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.838856  4426 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.841312 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:09.843298 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:09.795888 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:09.791800  4423 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.809194 31140 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:09.820724 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:09.821398  4521 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:09.822120  4443 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.847990 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:09.847735 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51893.8us
I20260709 12:26:09.849112 31140 raft_consensus.cc:2410] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:09.822969  4524 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.823421  4552 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.853657  4566 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:09.854374 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.855258 31140 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:09.856287 31140 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:26:09.856529  4530 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 12:26:09.840307  4541 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.856585 31140 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:09.848703  4027 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:09.851168 31689 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:09.871515 31689 raft_consensus.cc:606] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:09.871184 31140 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.871838 31689 raft_consensus.cc:614] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:09.872067 31140 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.852826  4515 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:09.850338 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:09.856529  4491 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.867571  4535 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:09.765633  4554 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.868527  4345 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:09.869357 30957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.014s	sys 0.037s
W20260709 12:26:09.880990 30957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.015s	sys 0.038s
W20260709 12:26:09.881605 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.881671 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 61021.5us
I20260709 12:26:09.765419  4509 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.852227  4418 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:09.861548 31141 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:26:09.873641 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:09.886783 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2634) took 1956 ms. Trace:
I20260709 12:26:09.887238 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:09.861807  4387 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.932s	user 0.008s	sys 0.022s
I20260709 12:26:09.884256 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:09.803906  4380 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.885896 31140 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:26:09.848927  4522 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:09.818547  4542 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:26:09.887132 31141 rpcz_store.cc:276] 0709 12:26:07.944976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.181667 (+236691us) service_pool.cc:224] Handling call
0709 12:26:08.224120 (+ 42453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.833069 (+1608949us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b54003c1f20. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb80bdc0 00007f2eeb8961c0 00007f2eeb897231 00007f2eeb893d8f 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c
0709 12:26:09.886561 (+ 53492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.886734 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4636928}
I20260709 12:26:09.883661  4566 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:09.899039 31141 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:09.901288 31141 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:09.892040 31140 pending_rounds.cc:85] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 13
I20260709 12:26:09.880016  4535 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:09.889794  4387 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.960s	user 0.008s	sys 0.025s
I20260709 12:26:09.902145 31140 pending_rounds.cc:107] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:26:09.922344 31140 raft_consensus.cc:2889] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.888849  4418 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.903306 31694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:09.880860  4027 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:09.911264 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.911405 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.912119 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.932706 31694 raft_consensus.cc:606] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:09.933314 30989 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:09.913118 31141 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.912118 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.915215 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.914582 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:09.934099 31141 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:09.916601 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.920727 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.886879  4423 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:09.919613 31064 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 12:26:09.903174 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:09.877300  4543 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:26:09.933457 31694 raft_consensus.cc:614] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:09.897953  4463 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:09.891296  4491 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:09.895728  4419 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.888059  4506 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:09.897830  4541 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 12:26:09.891409  4420 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:09.989900  4491 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:09.991485  4420 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.990178  4419 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.935920  4423 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.992661  4423 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:09.993573  4419 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:09.989845 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.997675 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.939016 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.998088  4544 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:09.999403 30984 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:10.000325  4544 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.000138  4423 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.000908 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.936817  4418 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.002355  4423 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:09.940380 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.004284  4554 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.004231 31068 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:10.007296 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:10.007977 31060 raft_consensus.cc:2393] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:10.011013  3591 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:26:10.011332  4509 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.011518 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.012171 31137 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
W20260709 12:26:10.012445  4558 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:26:10.013206 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.014843 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1095) took 1601 ms. Trace:
I20260709 12:26:10.015326 31138 rpcz_store.cc:276] 0709 12:26:08.413929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.519567 (+1105638us) service_pool.cc:224] Handling call
0709 12:26:09.519925 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.520315 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.520352 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.526938 (+  6586us) log.cc:844] Serialized 26 byte log entry
0709 12:26:09.702143 (+175205us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:09.702183 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.702270 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.014595 (+312325us) raft_consensus.cc:1685] finished
0709 12:26:10.014616 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.014770 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1999,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12650,"thread_start_us":146632,"threads_started":1,"wal-append.queue_time_us":343790}
I20260709 12:26:10.017496 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:09.923270 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1114) took 1273 ms. Trace:
W20260709 12:26:09.938575  4511 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:26:09.902472  4388 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:26:09.946769 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:09.946780 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.026657 30992 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:09.952919 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:09.953035  4057 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:09.954465  4424 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:09.901371  4522 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:09.931361  4535 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.935611 31065 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:10.014272  4565 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:26:09.930761  4569 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:26:09.902107  4547 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:09.987375 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.082351 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:09.991945  4491 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:10.018200 31138 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:10.019116 31064 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:10.019382 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.019510 31140 rpcz_store.cc:276] 0709 12:26:08.649717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.855830 (+1206113us) service_pool.cc:224] Handling call
0709 12:26:09.856162 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.922997 (+ 66835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.923218 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.083503 31137 raft_consensus.cc:2376] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16. Re-sending same reply.
I20260709 12:26:10.083847 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.083503 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.084628 31140 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20.
W20260709 12:26:09.941035 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 61844.2us
I20260709 12:26:10.021875 30985 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:10.085005 31691 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:10.086004 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:10.087481 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.087724 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.033828 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:10.072002 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.023443  4572 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.133s	user 0.000s	sys 0.002s
I20260709 12:26:09.993587  4420 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.092068 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:10.092552  4570 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:10.078097 31135 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:10.078855 31060 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:10.079337 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.014675  4463 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:10.010259  4418 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.081739 31689 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:10.120157 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:10.121714 30983 raft_consensus.cc:2393] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:10.122025 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.126289 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.117937  3893 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:10.077119  4057 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.098645  4522 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:10.137446 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49445.1us
I20260709 12:26:10.121026  4463 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.140990 31138 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:10.142035 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.143095 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.092751 31137 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:10.146656 31138 raft_consensus.cc:2410] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:26:10.146898 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.147445  4176 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:10.148343 31137 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:10.148873 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1116) took 1498 ms. Trace:
I20260709 12:26:10.149149 31137 rpcz_store.cc:276] 0709 12:26:08.650193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.147755 (+1497562us) service_pool.cc:224] Handling call
0709 12:26:10.148071 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.148658 (+   587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.148814 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.947510  4027 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.111821 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:10.114792 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 195449us
W20260709 12:26:10.151003 31033 connection.cc:256] RPC call timeout handler was delayed by -1.2735385410001072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
I20260709 12:26:10.151749 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.089188 30955 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:10.115918 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.089183 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:10.023257 31141 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:26:10.153512 31141 pending_rounds.cc:85] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 11
I20260709 12:26:10.153769 31141 pending_rounds.cc:107] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 18.12
W20260709 12:26:10.152870 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 63770.4us
I20260709 12:26:10.153990 31141 raft_consensus.cc:2889] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:10.152892 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.153921 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:10.147384 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.154723 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1115) took 1504 ms. Trace:
I20260709 12:26:10.155612 31138 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:10.155602 31141 rpcz_store.cc:276] 0709 12:26:08.649950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.898513 (+1248563us) service_pool.cc:224] Handling call
0709 12:26:09.898895 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.154516 (+255621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.154677 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.152789 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:10.159044 31137 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:10.163782 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:10.164065  4509 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:10.166290  4563 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:10.166899  4563 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.167411  4563 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.168412  4554 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:10.163738  4543 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:10.170161  4543 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:10.166333  4575 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:10.132956  4418 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.172299 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.172964 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.136718  4380 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:10.177695 31061 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:10.173250 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.180052 31064 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:10.180086 31698 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
W20260709 12:26:10.181587 31033 leader_election.cc:336] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.581s (SENT)
I20260709 12:26:10.183881 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.184551 31064 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:10.185446 31138 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:26:10.087246  4547 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.189154 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.187819 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:10.192785 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:10.172776  3893 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:10.186780  4388 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.203428  4388 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.204149  4544 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:10.205857  4544 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:10.139586  4545 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.208390  4543 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.214191 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:10.171475  4382 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:10.202672  4420 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:10.217844 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:10.216732 31061 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:10.220561 31062 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:10.221184  4568 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:10.222945 31064 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:26:10.223068 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:10.224509  4554 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:10.221882  4541 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:26:10.218765  4345 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:10.226773  4345 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.227224  4345 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:10.226840  4549 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.229017  4549 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:10.226338  4509 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:26:10.229995  4388 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.231195 31138 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:10.190320  4387 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:10.246237 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:10.237623  4552 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:10.256458 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.257030 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:10.258318 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:10.259594 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:10.245491  4270 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:10.262836 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.263988 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:10.263733  4541 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:10.282828 30989 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:10.283092 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.283646 31144 raft_consensus.cc:2376] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16. Re-sending same reply.
I20260709 12:26:10.284544 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.287655  4263 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.259845  4435 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:26:10.266384  4561 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:10.288702  4263 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.288936  4561 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:10.289376  4561 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.291210 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.291791 31142 raft_consensus.cc:2376] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19. Re-sending same reply.
I20260709 12:26:10.292872 31142 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:26:10.298436 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.298873 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.298909  4547 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:10.299326 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.299853  4057 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.302130  4382 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.285319 31144 raft_consensus.cc:2410] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:26:10.302770  4382 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.249850  4540 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:26:10.306957  4382 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.319386  4452 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.319905  4452 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.264159  4387 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:10.323428  4387 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.245769  4524 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:26:10.323751  4057 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.324282  4057 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.323777  4387 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.314044 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2953) took 2692 ms. Trace:
I20260709 12:26:10.330574 31062 rpcz_store.cc:276] 0709 12:26:07.621511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.833359 (+211848us) service_pool.cc:224] Handling call
0709 12:26:08.236521 (+403162us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5800129398. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:26:08.236698 (+   177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.221017 (+1984319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.313783 (+ 92766us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54000c7454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:10.313962 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5236716672}
I20260709 12:26:10.256186  4552 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:10.258392  4424 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.331700 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:10.328799 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:10.337898 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:10.339011 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:10.322047  4452 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.265129  4270 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:10.334441  4057 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:10.370698  4563 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.371213  4535 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.337586  4552 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:10.377491 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.379869 31063 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:10.366685  4435 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:10.382511  4561 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.347873  4270 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.367792  4424 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:10.344754 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:10.482684 31664 connection.cc:256] RPC call timeout handler was delayed by -1.919918539000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21030000000000004s for a response
W20260709 12:26:10.483060 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 138402us
W20260709 12:26:10.483238 31664 connection.cc:256] RPC call timeout handler was delayed by -1.7482473610002671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2731s for a response
W20260709 12:26:10.483687 31664 connection.cc:256] RPC call timeout handler was delayed by -1.4915185390000261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2579s for a response
W20260709 12:26:10.483960 31664 connection.cc:256] RPC call timeout handler was delayed by -1.4867301919998681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
W20260709 12:26:10.484229 31664 connection.cc:256] RPC call timeout handler was delayed by -1.4867301919998681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
W20260709 12:26:10.484542 31664 connection.cc:256] RPC call timeout handler was delayed by -1.4743877959999736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
W20260709 12:26:10.484812 31664 connection.cc:256] RPC call timeout handler was delayed by -1.3234072129998822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3368s for a response
I20260709 12:26:10.368474  4547 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:10.375598  4508 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.386320  4526 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.488927 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.506980 31696 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
W20260709 12:26:10.507597 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.490015 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:10.508080 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.494093 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.496347 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.498270 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.500311 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:10.502118 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48350.3us
I20260709 12:26:10.509740 31692 raft_consensus.cc:2410] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 12:26:10.490379 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.511101 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.511829 31689 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:10.509735 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.507126  4359 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:10.514398  4359 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:10.515507  4561 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.514052  4526 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.515268  4359 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.520555 30987 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:10.523833 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.525935 31693 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:10.528154 31689 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 12:26:10.534327 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.457152  4435 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.536999 31068 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:10.502573  4057 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.543460 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:10.544442 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.538883 30984 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:10.503739  4547 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:10.507092  4452 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.558090 30983 raft_consensus.cc:2376] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16. Re-sending same reply.
I20260709 12:26:10.518764  4511 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:26:10.518458  4529 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:26:10.600273 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:10.554411  4435 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:10.614209 30991 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.572239  4547 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.586267  4176 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.551643  4339 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:26:10.572861  4057 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:10.656173  4532 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.254s	user 0.003s	sys 0.000s
I20260709 12:26:10.656250  4057 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:26:10.661813  4468 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.662735 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.663137  4563 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.663851 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:10.664247  4535 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.664798  4535 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:10.665400  4535 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:10.665727  4535 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:10.670583 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.671677 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.672221 31138 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:10.672933 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.583339  4549 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:26:10.674540 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:10.678273  4588 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:26:10.679811 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.680788 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:10.681427  4491 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.588s	user 0.005s	sys 0.002s
I20260709 12:26:10.674582  4549 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:10.683553  4549 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:10.684290  4549 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:10.684809  4463 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.513s	user 0.003s	sys 0.004s
I20260709 12:26:10.685446  4549 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:10.689178 31138 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:10.690274 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.690500 31059 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:10.691002 31138 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:10.681813 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.691690 31144 raft_consensus.cc:2376] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20. Re-sending same reply.
I20260709 12:26:10.689026 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:10.700531 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:10.702280 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:10.703291 31059 raft_consensus.cc:2410] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:10.691697 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.707739 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:10.709326 31138 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:10.710754 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:10.716885 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.685185  4463 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.666091  4535 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:10.681810  4491 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.719445  4390 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:10.719601 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.760736 31138 raft_consensus.cc:2376] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17. Re-sending same reply.
I20260709 12:26:10.761701 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.762655 31138 raft_consensus.cc:2410] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:26:10.723453 31136 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:10.730767  4445 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.726131  4526 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:10.734772  4585 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 12:26:10.739912 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.711987  4418 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:10.742537 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:10.744009 31143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 1109) took 2095 ms (client timeout 2579 ms). Trace:
W20260709 12:26:10.772027 31143 rpcz_store.cc:269] 0709 12:26:08.648481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.376085 (+727604us) service_pool.cc:224] Handling call
0709 12:26:10.743759 (+1367674us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54003f43d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:10.743925 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":585714560}
I20260709 12:26:10.772748 31064 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:10.642550  4176 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.746992 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.780187 31140 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:10.780711 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.781066 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.781303 31143 raft_consensus.cc:2376] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12. Re-sending same reply.
I20260709 12:26:10.782256 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:10.784097  4596 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:10.782779 31143 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:10.754069 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:10.755106 31033 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20.
I20260709 12:26:10.786286 31033 leader_election.cc:403] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:10.786298 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.785996 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.787371 31140 raft_consensus.cc:2410] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:10.787529 31143 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:10.788177 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.788429 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.789052 31143 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:10.594709  4452 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:10.764084 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 63647.4us
I20260709 12:26:10.766557 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:10.769302 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.740232  3893 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:10.793828 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:10.744831  4463 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.789184 31064 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:10.807209 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.811913 30984 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:10.812950 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.814414 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.815528 30984 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:10.823299 31697 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:10.790462 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.757087 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.630750  4345 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.789052 31140 raft_consensus.cc:2410] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:26:10.772912  4418 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.838327  4418 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.794773 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:10.789791  4529 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.794483  3893 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:10.794538  4468 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20.
I20260709 12:26:10.804833 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.805524 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.847067 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:10.849066 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:10.850235 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.792048 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:10.831769 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.832089 31144 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:10.835121 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:10.791363  4263 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:10.839497 30992 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.840533 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.853474 30984 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.853969 31140 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:10.842434 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.855262 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.856818 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:10.859902 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:10.860877 31142 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:10.860929 30986 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:10.857046 31143 raft_consensus.cc:2410] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 12:26:10.865600 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73635.4us
W20260709 12:26:10.867620 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 329457us
I20260709 12:26:10.866357 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:10.871918 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:10.868395 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.879818 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:10.881682 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:10.884737 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:10.886855 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:10.864410  4463 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:10.863083  4543 raft_consensus.cc:2764] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:26:10.893553  3893 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:10.897162  4387 consensus_meta.cc:287] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.000s
I20260709 12:26:10.864643  4393 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:10.898977  4176 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.900774  4345 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.901582  4491 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.843173  4529 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.878548  4600 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:10.878984  4565 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:10.902729  4593 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:26:10.903867  4390 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:10.921128 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 69517.5us
I20260709 12:26:10.877089  4544 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:10.864753  4263 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.907764 30992 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:10.908726 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:10.898053  4270 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:10.912048 30984 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:10.864586  4445 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
W20260709 12:26:10.910210 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 237206us
I20260709 12:26:10.914660 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:10.912731  4526 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:10.905573  4435 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.940505 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:10.942618 30986 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:26:10.921267  4388 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:10.925154  4544 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:10.778133 30991 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:10.913007  4452 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:10.862391  4594 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:10.954792  4594 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:10.955307  4594 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.963282  4529 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:10.965161  4545 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:10.913852  4393 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:10.965914  4393 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:10.918668  3893 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.929623  4508 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:10.971801  4508 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.972311  4508 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.930028  4443 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:10.974452  4443 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:10.975883  4388 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:10.975579  4443 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.977046  4443 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:10.927372  4263 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:10.978775  4601 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:10.921416  4390 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.979839  4390 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:10.932721 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2560) took 1365 ms. Trace:
I20260709 12:26:10.982368 30988 rpcz_store.cc:276] 0709 12:26:09.567313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.567410 (+    97us) service_pool.cc:224] Handling call
0709 12:26:09.567737 (+   327us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:09.648511 (+ 80774us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:09.701302 (+ 52791us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.327267 (+625965us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b180037f420. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:26:10.578033 (+250766us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:10.579596 (+  1563us) log.cc:844] Serialized 78 byte log entry
0709 12:26:10.579787 (+   191us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:10.579832 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.579921 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.926170 (+346249us) raft_consensus.cc:1685] finished
0709 12:26:10.932467 (+  6297us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.932650 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35936,"prepare.queue_time_us":52627,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":10014,"raft.queue_time_us":95438,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":10545,"spinlock_wait_cycles":3852544,"thread_start_us":282969,"threads_started":2,"wal-append.queue_time_us":1065622}
I20260709 12:26:10.984651 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.985364  4339 raft_consensus.cc:993] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:10.985729 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.934516 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:10.952726 30987 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:10.986183  4544 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:10.988539 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.976399  4388 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:10.996256 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.991671 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.992300 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.997004 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2655) took 1078 ms. Trace:
I20260709 12:26:10.997917 31135 rpcz_store.cc:276] 0709 12:26:09.918650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.000625 (+ 81975us) service_pool.cc:224] Handling call
0709 12:26:10.996788 (+996163us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b5400062f54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:10.996929 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175871232}
I20260709 12:26:10.992899 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:10.993428 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.999287 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:10.999909 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.000102 31135 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:10.999295 31692 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:10.990553  4535 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:10.986634 30992 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:26:10.919663  4428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.740s	user 0.023s	sys 0.027s
I20260709 12:26:10.996974 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.001346 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:11.006177 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.012658 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:11.014050 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.014084 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:11.015379 31135 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:11.015318 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.017047 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:11.017864 30991 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:11.018465 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:11.021229 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 83634.4us
I20260709 12:26:11.023268 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:10.991978 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:11.006922 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:11.024731 31698 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:11.006839 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.927246  4543 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:26:10.919490  4387 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.010862 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:11.025775 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:11.009503 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:11.035640 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:10.942131  4345 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:26:11.026237 31144 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:11.037503 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2561) took 1469 ms. Trace:
I20260709 12:26:11.037986 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:11.017706  4428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.839s	user 0.023s	sys 0.028s
I20260709 12:26:11.056140 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:11.039183 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:10.994714  4270 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:11.044492 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.059870 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:11.060303 30985 raft_consensus.cc:2410] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:10.969686  4418 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.043066 30990 rpcz_store.cc:276] 0709 12:26:09.567581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.567684 (+   103us) service_pool.cc:224] Handling call
0709 12:26:11.037277 (+1469593us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002c1d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:11.037408 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374971648}
I20260709 12:26:10.942955  4547 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.026930  4527 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:11.046577 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:11.068446 31144 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:26:11.049422 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:11.050091 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:11.069360 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 199144us
I20260709 12:26:11.069682 31695 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:11.048650  4508 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.050516 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:11.050767 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.072889 30991 raft_consensus.cc:2368] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13.
I20260709 12:26:11.073566 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.040052  4051 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:11.051152 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:11.074270 30985 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:26:11.052876  4387 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.011308  4545 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:10.969595  4594 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.059449  4549 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:11.054320 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:11.053121 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:11.084573 31696 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:11.085439  4545 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:26:11.086730  4589 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.432s	user 0.002s	sys 0.002s
I20260709 12:26:11.086587  4545 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.053759 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:11.063261 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.035046  4359 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:11.090458  4359 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.090966  4051 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:11.092680  4359 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.035064 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.094877 30992 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:11.096063 30992 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:26:11.097159 30992 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:11.035260  4420 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:11.074975 30990 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:11.098759  4420 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:11.074124 30991 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:11.099707  4420 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.075490 30985 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:11.016846  4600 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:26:11.107491 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:11.108819  4531 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:11.110527 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:11.111109 31144 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:11.111289 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1555) took 1333 ms. Trace:
I20260709 12:26:11.111629 31060 rpcz_store.cc:276] 0709 12:26:09.778201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.008495 (+230294us) service_pool.cc:224] Handling call
0709 12:26:10.078540 (+ 70045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.099720 (+ 21180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.111106 (+1011386us) spinlock_profiling.cc:243] Waited 69.7 ms on lock 0x7b54002a3850. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:11.111238 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160367488}
I20260709 12:26:11.117203 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 884) took 1101 ms. Trace:
I20260709 12:26:11.118582 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:11.119169 31062 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:11.119460 31064 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:11.120182 31067 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:11.011123  4552 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.072070  4501 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:11.076705 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.059152  4511 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.053879 31692 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:11.082401 30983 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:11.068179  4423 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:11.117576 31694 rpcz_store.cc:276] 0709 12:26:10.016197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.016344 (+   147us) service_pool.cc:224] Handling call
0709 12:26:10.016617 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.155727 (+139110us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:10.155939 (+   212us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.507328 (+351389us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.507396 (+    68us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:10.507447 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.117016 (+609569us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540081e7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:11.117056 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.117155 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49780,"prepare.queue_time_us":9840,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":50171,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":12408,"spinlock_wait_cycles":355223680,"thread_start_us":301252,"threads_started":1,"wal-append.queue_time_us":526508}
I20260709 12:26:11.064899  4528 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:11.056283  4597 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:26:11.128086 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:11.130028 31143 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:26:11.074672  4605 raft_consensus.cc:1081] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:11.138044 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:11.091033 31109 leader_election.cc:400] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13.
I20260709 12:26:11.146672 30992 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 20. Ops: [18.12-18.12]
I20260709 12:26:11.064318  4526 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
W20260709 12:26:11.150647 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 214863us
I20260709 12:26:11.182731 31109 leader_election.cc:403] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:11.183645 31062 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:26:11.184269 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.078994 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:11.185724 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 106799us
I20260709 12:26:11.185868 31062 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:26:11.186741 31143 pending_rounds.cc:85] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 9
I20260709 12:26:11.187072 31143 pending_rounds.cc:107] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:26:11.187362 31143 raft_consensus.cc:2889] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:11.189349 31060 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:11.130297  4527 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:11.194782 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 103897us
I20260709 12:26:11.146807  4423 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:11.215329 30985 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:11.216147 31139 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:11.211109 31067 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:11.211279  4051 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:11.069485  4393 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.220436 31065 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:11.221419  4594 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.222332  4393 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:11.229753  4552 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.191244  4563 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:11.216468 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.231153 30992 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:11.232232  4360 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:11.234444  4612 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:11.238142  3893 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:11.239050  4508 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.239149  4549 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:11.238793  4388 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.239600  4549 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:11.240234  4418 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.215348  4435 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.242415 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.248232 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:11.249055 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:11.250936 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.160816  4511 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.268414 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:11.270785 30989 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:11.184638  4531 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:26:11.279493 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:11.273046 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:11.276261 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:11.276778 31061 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:11.274706 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:11.282337 31060 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:11.295229 31060 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:11.144636  4359 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.300299 31139 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:26:11.258666  4569 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.004s	sys 0.006s
I20260709 12:26:11.301810 31064 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:11.303148 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:11.303956 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:11.255360  4508 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:11.315083 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 234660us
I20260709 12:26:11.305474  4544 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.215575  4527 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.315047  4359 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.336127 30989 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:11.348762 31139 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:11.349535 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1120) took 1005 ms. Trace:
I20260709 12:26:11.349939 31139 rpcz_store.cc:276] 0709 12:26:10.343876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.347972 (+1004096us) service_pool.cc:224] Handling call
0709 12:26:11.348370 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.349253 (+   883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.349473 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.353875 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.355357 31139 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:11.314458  4569 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.005s	sys 0.006s
I20260709 12:26:11.364313 30988 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:11.369140 31139 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:26:11.369637 31139 pending_rounds.cc:85] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 9
I20260709 12:26:11.369925 31139 pending_rounds.cc:107] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:26:11.268244  4390 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.382004 31139 raft_consensus.cc:2889] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:11.382326 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.317354  4428 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:26:11.309846  4501 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:11.420321 30989 raft_consensus.cc:2410] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:11.323066  4526 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:26:11.361166  4508 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:11.357741  4535 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.322854  4263 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:11.382345 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 131522us
W20260709 12:26:11.356695 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 334609us
I20260709 12:26:11.312196  4423 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.433339 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:11.418387  4359 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.427774 31139 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:26:11.435250 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.437094 31139 pending_rounds.cc:85] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 7
I20260709 12:26:11.437436 31139 pending_rounds.cc:107] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:26:11.425173  4501 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.441344 31143 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:11.441617 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:11.438700 31139 raft_consensus.cc:2889] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:11.428467 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 107219us
I20260709 12:26:11.421135  4600 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:26:11.433064  4435 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:11.445716 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 2965) took 1724 ms. Trace:
I20260709 12:26:11.452076 31068 rpcz_store.cc:276] 0709 12:26:09.721488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.005565 (+284077us) service_pool.cc:224] Handling call
0709 12:26:10.536814 (+531249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.540373 (+  3559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.445542 (+905169us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b54001a77d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:11.445662 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221121408}
I20260709 12:26:11.444025 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.456475 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.461047 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:11.462961 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:11.464345 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:11.467772 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.468547 30989 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:11.440830  4423 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:11.442735  4501 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:11.435636  4605 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
W20260709 12:26:11.502534 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 67348.1us
W20260709 12:26:11.560003 31136 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2475) took 1965 ms (client timeout 2035 ms). Trace:
W20260709 12:26:11.613607 31136 rpcz_store.cc:269] 0709 12:26:09.594283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.651145 (+ 56862us) service_pool.cc:224] Handling call
0709 12:26:11.559798 (+1908653us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540029e5d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:11.559923 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2370454272}
I20260709 12:26:11.537268 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:11.469671  4526 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:11.575047 31138 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
W20260709 12:26:11.565547 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 219272us
I20260709 12:26:11.461172  4565 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:11.617710 31058 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:11.520412 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 64039.9us
I20260709 12:26:11.620882 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.466004  4618 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:11.572575  4620 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:26:11.466861  4359 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:11.473856  4638 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.095s	user 0.003s	sys 0.000s
W20260709 12:26:11.625856 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.573720  4051 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 12. Current term is 13.
I20260709 12:26:11.629860 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.529862  4491 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:11.535526  4644 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.497833  4621 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:26:11.578802  4418 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:11.537740  4636 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:11.634842 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:11.593731  4544 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.497550  4527 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:11.528390  4511 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:11.567476  4635 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:11.627887 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.633675 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 178033us
I20260709 12:26:11.466347  4435 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.617309  4526 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:11.653612 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2669) took 1384 ms. Trace:
I20260709 12:26:11.475018  4420 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.153s
I20260709 12:26:11.654314 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:11.620599  4565 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:11.503525  4388 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:11.635784 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 171509us
I20260709 12:26:11.627662 31064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48047b2b139146c38a821328cf7bd148"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:26:11.624311  4535 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:11.642103  4630 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:26:11.649466 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:11.633734  4542 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:11.654052 31144 rpcz_store.cc:276] 0709 12:26:10.269474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.111726 (+842252us) service_pool.cc:224] Handling call
0709 12:26:11.112028 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.112319 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.112343 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.112742 (+   399us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.299234 (+186492us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:11.299296 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.306003 (+  6707us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.653415 (+347412us) raft_consensus.cc:1685] finished
0709 12:26:11.653433 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.653558 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1036,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"thread_start_us":186188,"threads_started":1,"wal-append.queue_time_us":185139}
I20260709 12:26:11.661392 31064 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:11.687407 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.691376 31064 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:11.693317  4514 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:11.693750  4514 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:11.692940 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:11.688012 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 50657.5us
I20260709 12:26:11.637754  4418 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:11.692924 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:11.639560  4618 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 12:26:11.700133 31064 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.711391 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2496) took 1376 ms. Trace:
I20260709 12:26:11.711912 31140 rpcz_store.cc:276] 0709 12:26:10.337379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.113707 (+776328us) service_pool.cc:224] Handling call
0709 12:26:11.113992 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.114318 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.114348 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.114753 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.357489 (+242736us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:11.375095 (+ 17606us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.375275 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.375347 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.711129 (+335782us) raft_consensus.cc:1685] finished
0709 12:26:11.711152 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.711320 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17105,"prepare.queue_time_us":25002,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":79422,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":24565,"thread_start_us":242399,"threads_started":1,"wal-append.queue_time_us":241980}
I20260709 12:26:11.637316  4597 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:26:11.685371  4612 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.753206 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 60400.2us
I20260709 12:26:11.694156  4514 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:11.753937 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:11.751350 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 118608us
I20260709 12:26:11.635262  4648 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:26:11.755702 31058 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:11.653892  4623 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:26:11.706586  4418 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:11.646212  4628 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.114s
I20260709 12:26:11.756749 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:11.635537  4644 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.706036  4452 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:11.643527  4642 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 12:26:11.744556  4607 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:11.696262  4531 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:26:11.769351 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:11.756489 31064 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:11.754760  4641 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:11.769279  4420 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:11.754168  4526 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:11.883560  4463 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:11.884101  4463 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:11.886777  4644 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:11.887409  4621 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:26:11.888412  4653 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:26:11.890156  4624 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:11.890576  4624 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:11.891446  4624 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:11.831786  4650 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 12:26:11.822055 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 52785.5us
I20260709 12:26:11.768414  4388 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:11.879364  4443 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.902s	user 0.013s	sys 0.001s
I20260709 12:26:11.831354  4428 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:11.831260  4535 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:11.911772 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:11.793780  4542 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 12:26:11.884153  4526 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:11.787388  4612 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.892742  4624 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:11.913522  4428 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.932333  4428 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:11.935459  4514 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.937721  4514 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:11.881105  4544 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:26:11.950001 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2818) took 1085 ms. Trace:
I20260709 12:26:11.950591 30986 rpcz_store.cc:276] 0709 12:26:10.865087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.868175 (+  3088us) service_pool.cc:224] Handling call
0709 12:26:11.949784 (+1081609us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b54000762d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:11.949932 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151968128}
I20260709 12:26:11.926811  4535 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:11.959066  4535 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:11.963477  4606 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.857s	user 0.000s	sys 0.002s
I20260709 12:26:11.966234 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2502) took 1627 ms. Trace:
I20260709 12:26:11.966748 31143 rpcz_store.cc:276] 0709 12:26:10.339116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.188232 (+849116us) service_pool.cc:224] Handling call
0709 12:26:11.188471 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.441736 (+253265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.965920 (+524184us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540006fed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:11.966141 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581158784}
I20260709 12:26:11.969131 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1559) took 1295 ms. Trace:
I20260709 12:26:11.970526  4428 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.969630 31063 rpcz_store.cc:276] 0709 12:26:10.673990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.675433 (+  1443us) service_pool.cc:224] Handling call
0709 12:26:10.675730 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.676044 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.676071 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.676527 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.919708 (+243181us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:10.919753 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.919873 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.962650 (+1042777us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:11.968756 (+  6106us) raft_consensus.cc:1685] finished
0709 12:26:11.968786 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.969058 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5925,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":50334464,"thread_start_us":242784,"threads_started":1,"wal-append.queue_time_us":404474}
I20260709 12:26:11.972615  4428 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:11.973215 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 61574.9us
I20260709 12:26:11.975086  4554 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:11.975430  4382 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:11.975478  4554 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:11.977612  4554 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:11.979879  4634 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.379s	user 0.000s	sys 0.001s
W20260709 12:26:11.950209  4590 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.267s	user 0.001s	sys 0.002s
W20260709 12:26:11.983466  4619 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.552s	user 0.001s	sys 0.000s
I20260709 12:26:11.985541  4554 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:11.986004 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2666) took 1723 ms. Trace:
I20260709 12:26:11.986387 31137 rpcz_store.cc:276] 0709 12:26:10.262541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.108369 (+845828us) service_pool.cc:224] Handling call
0709 12:26:11.108669 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.108991 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.109021 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.109422 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.346463 (+237041us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:11.346499 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.346561 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.982791 (+636230us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:11.985737 (+  2946us) raft_consensus.cc:1685] finished
0709 12:26:11.985757 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.985948 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4182,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":81,"spinlock_wait_cycles":3077376,"thread_start_us":236617,"threads_started":1,"wal-append.queue_time_us":272657}
I20260709 12:26:11.988590  4636 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:11.989115  4636 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:11.991329 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.994091 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:12.000344 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:12.001251 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:11.992520 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:12.023365 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.023931 31138 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2482) took 1716 ms (client timeout 1940 ms). Trace:
W20260709 12:26:12.024673 31138 rpcz_store.cc:269] 0709 12:26:10.307674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.765980 (+458306us) service_pool.cc:224] Handling call
0709 12:26:12.023731 (+1257751us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b540012ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:12.023863 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800801152}
W20260709 12:26:12.026492 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 218162us
I20260709 12:26:12.031236 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.031922 31143 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:11.998718  4491 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.000888  4511 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.000864  4382 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.036372 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.000887  4527 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.023319  3893 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.957973  4501 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.044186 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.062418 31137 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:12.045150 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.051221 31063 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:12.036880 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:12.060678 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59511.8us
I20260709 12:26:12.026234  4435 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:12.033517  4654 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:12.053239 31143 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:12.022775  4528 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:26:12.066041 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:12.074857 31138 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:12.059409  4382 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.095654  4554 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.095971  4643 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:12.096312  4388 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:12.096585  4643 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:12.064891  3893 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.094727  4428 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:26:12.098573  3893 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.049183  4418 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:12.084637  4649 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 12:26:12.092360 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:12.092588 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:12.096624  4382 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:12.082402  4435 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:12.060249  4307 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.065101  4359 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:12.105135 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:12.064877  4612 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:12.109694  4662 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:26:12.048213  4491 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.119963 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.116230 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:12.116742 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:12.115830 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:12.114274 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.097605  4643 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:12.249920 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 135393us
I20260709 12:26:12.051066  4511 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.119321  4307 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.264745  4263 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:12.265581  4263 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.265596  4307 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.114578 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:12.180871 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.275450 31140 raft_consensus.cc:2410] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:26:12.063409  4527 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.294919 31698 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:12.257203 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.266083  4263 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:12.268641  4645 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.516s	user 0.000s	sys 0.000s
I20260709 12:26:12.267283  4642 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:12.117539 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:12.107990  4618 raft_consensus.cc:2764] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:12.102313  4654 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:12.250247  4382 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:12.387740  4382 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:12.388192  4382 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:12.121670  4435 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:12.390411  4435 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:12.390786  4655 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 12:26:12.390841  4435 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:12.392717  4468 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:12.393163  4468 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:12.122028  3893 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.393989  4468 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.394663  3893 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.395115  4468 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:12.254767 31136 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:12.254086 31139 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:12.255940 31142 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:12.113181  4651 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:12.305321 31137 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:12.110767  4359 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:12.315367 31664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.040s	sys 0.036s
W20260709 12:26:12.373660 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.398636 31664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.040s	sys 0.036s
I20260709 12:26:12.258069  4528 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.109767  4420 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.195s
I20260709 12:26:12.384704  4654 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.386178  4491 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:12.377179  4051 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.372535  4307 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:12.253755  4635 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.125s
W20260709 12:26:12.399178 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 706238us
I20260709 12:26:12.379395  4642 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:12.412024  3893 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:12.420027  4263 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:12.424029 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:12.420773 31698 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:12.424358 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:12.446734 31137 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:12.428165 31136 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:12.403862  4359 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:12.447949  4642 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.449229  4642 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:12.403978  4651 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.450606  4651 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.421681  4307 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.451956 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:12.452584  4307 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:12.430274 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:12.453056 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:12.383186  4595 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:12.453812  4628 raft_consensus.cc:2764] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:26:12.385886  4656 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.144s
I20260709 12:26:12.421448  4051 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.454828  4648 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:12.455363  4656 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:12.455763  4656 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:12.456310  4051 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.454572 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2657) took 2284 ms. Trace:
I20260709 12:26:12.456496  4656 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.430825 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:12.457542  4612 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:12.457873  4647 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:12.457907  4612 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.458349  4647 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:12.458585  4612 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:12.459645 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:12.461169 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:12.461839 30985 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:12.422986  4528 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.462849  4501 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.463320 30991 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:26:12.464347  4528 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.465732  4418 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.466212  4565 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.456943  4648 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:12.467278  4565 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:12.456876 31141 rpcz_store.cc:276] 0709 12:26:10.169683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.169821 (+   138us) service_pool.cc:224] Handling call
0709 12:26:10.170107 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.170462 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:10.257047 (+ 86585us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.546919 (+289872us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.576832 (+ 29913us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.576980 (+   148us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:10.622955 (+ 45975us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.628768 (+  5813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.628859 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.885241 (+1256382us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b1800408ee0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:12.090659 (+205418us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b1800408ee0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:12.454342 (+363683us) raft_consensus.cc:1685] finished
0709 12:26:12.454364 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.454513 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57962,"prepare.queue_time_us":249686,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":4944,"raft.queue_time_us":14368,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":43329,"spinlock_wait_cycles":109783680,"thread_start_us":294604,"threads_started":2,"wal-append.queue_time_us":473897}
I20260709 12:26:12.468950 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:12.454536  4628 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:26:12.473022 31142 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:12.445120  4420 consensus_queue.cc:1243] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:26:12.481621  4626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.051s	sys 0.030s
W20260709 12:26:12.482151  4626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.051s	sys 0.031s
I20260709 12:26:12.483428  4051 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.484082  4051 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.486822  4051 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:12.467303  4511 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:12.479447  4535 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.013s	sys 0.068s
I20260709 12:26:12.476373  4435 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.478271  4339 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:12.590662 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.007s	sys 0.006s
W20260709 12:26:12.654577 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.007s	sys 0.007s
W20260709 12:26:12.656517 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 619678us
I20260709 12:26:12.657155  4652 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:12.658227  4652 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.660576  4443 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:12.661496  4443 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.594990 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:12.664623  4443 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:12.664767 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 70287.4us
I20260709 12:26:12.597463 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:12.666889  4652 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.669082  4511 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:12.669600  4511 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:12.670045  4511 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:12.604022 30988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2822) took 1695 ms (client timeout 2015 ms). Trace:
W20260709 12:26:12.607951 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 106702us
W20260709 12:26:12.674422 30988 rpcz_store.cc:269] 0709 12:26:10.908086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.985510 (+ 77424us) service_pool.cc:224] Handling call
0709 12:26:12.603523 (+1618013us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540020fb54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:12.603816 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865376640}
I20260709 12:26:12.677021 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:12.615916 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2586) took 2040 ms. Trace:
I20260709 12:26:12.497257  4607 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:12.677553  4605 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:12.677992 30983 rpcz_store.cc:276] 0709 12:26:10.574946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.576108 (+  1162us) service_pool.cc:224] Handling call
0709 12:26:10.576363 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.148318 (+571955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.615293 (+1466975us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54002c2254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:12.615701 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163374464}
I20260709 12:26:12.678655  4664 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:12.680547  4605 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:12.509919  4435 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:12.504779  4535 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.013s	sys 0.068s
I20260709 12:26:12.523963  4614 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:12.524930  4635 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:12.684268  4614 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:12.684440  4635 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:12.526036  4636 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.531948  4647 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.529373  4648 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:12.540779  4651 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.689898 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:12.692380  4390 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:12.692512 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:12.692735  4651 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:12.537324  4656 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:12.686117  4614 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.644826  4673 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.694809  4674 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:12.644192 31107 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.009s
W20260709 12:26:12.699909 31107 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.000s	sys 0.010s
I20260709 12:26:12.686508  4635 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.713327  4604 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:12.716773 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.718142  4614 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:12.726701  4635 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:12.731551  4547 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:12.732913  4648 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:12.740481 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.741412 31068 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:12.746372  4673 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.747936  4463 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.749469  4673 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.750618 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:12.735363 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:12.772305 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 68830.7us
I20260709 12:26:12.774752  4051 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:12.775278  4051 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.775692  4051 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:12.777495 31068 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:12.780359 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:12.782040 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.782176  4463 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.783869 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.785589 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.786715 30983 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.788208 31067 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:12.751118 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.751677 31106 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:12.792115 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:12.793133 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 54673.1us
I20260709 12:26:12.793442 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:12.751817 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.794637 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.757354  4677 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:12.794653  4263 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:12.795987  4677 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.796756 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:12.758289 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 66068.2us
I20260709 12:26:12.798878 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.800347 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:12.758390 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.802398 31140 raft_consensus.cc:2368] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:12.759621 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:12.760248 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.808422 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.808519  4628 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:12.810374  4677 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:12.760923  4527 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.738520  4604 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:12.762681 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.763021  4651 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:12.815434  4651 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.815642  4604 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.817826  4651 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:12.818586 31065 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:12.750053  4549 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.765697 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:12.769258 31690 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:12.741360  4674 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:12.802875 30987 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:26:12.819171 31691 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:12.823832 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.824090 31068 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:12.824580 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:12.825289 31692 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:12.831614 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:12.833376  4549 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.824086 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:12.849296 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44032.8us
I20260709 12:26:12.830951 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.868487  4027 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:26:12.870025  4027 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:12.870446  4027 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:12.871048 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.871600  4027 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.872458 31140 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:12.872645 31138 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:12.873481  4027 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:12.874428  4648 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:12.874989 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.875386  4648 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:12.875742 31136 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:12.875844  4648 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:12.877055  4648 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:12.880098 31143 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:26:12.881022  4648 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:12.882653 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.883476 31137 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:12.830904 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:12.887274 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 594087us
I20260709 12:26:12.887678 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:12.888501 31138 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:12.892638 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.899577 31141 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:12.899885 31692 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:12.907099 31697 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:12.910027 30983 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:12.916072 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.916435 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:12.914256 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:12.896795  4614 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:26:12.939139 31140 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:12.941093 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:12.942867 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:12.946240 31691 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.946743 31060 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:12.951071 31690 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 9.
I20260709 12:26:12.931623  4527 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:12.962558 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:12.955013 31064 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:13.001609 31697 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:13.003813 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:13.004228 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.007982 31062 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:13.020833 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.027858 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:13.028450 31060 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:13.032709 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:13.032945 31139 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:26:13.033397 31139 pending_rounds.cc:85] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 10
I20260709 12:26:13.033555 31068 raft_consensus.cc:2393] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 12: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:13.033658 31139 pending_rounds.cc:107] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:26:13.034129 31139 raft_consensus.cc:2889] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:13.035388 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.035905 31139 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:13.038941  4382 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.009s
I20260709 12:26:13.039408  4382 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.040400 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:13.040755 31691 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:13.018246  4359 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.042456 31067 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:13.123778 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2829) took 1539 ms. Trace:
W20260709 12:26:13.068578 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.142194 31136 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
W20260709 12:26:13.147908 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.159538 31137 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:13.080350  4359 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:13.160837  4686 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.056s	user 0.003s	sys 0.023s
I20260709 12:26:13.085628  4543 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.161780  4543 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.068737  4656 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.163203  4656 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.164534  4543 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:13.095207  4612 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.014s
I20260709 12:26:13.093590  4527 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:13.111706  4624 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.180s	user 0.006s	sys 0.011s
I20260709 12:26:13.061404 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:13.094663  4674 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:13.171703 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.145617 30987 rpcz_store.cc:276] 0709 12:26:11.584503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.584649 (+   146us) service_pool.cc:224] Handling call
0709 12:26:11.584986 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.882295 (+1297309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.123512 (+241217us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000767d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:13.123723 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2799410944}
I20260709 12:26:13.054090  4463 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:13.070302  4545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.825s	user 0.016s	sys 0.003s
W20260709 12:26:13.175643  4545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.083s	user 0.017s	sys 0.003s
I20260709 12:26:13.175577  4463 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.165707  4612 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.043668  4673 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:13.206760 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.041455  4382 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:13.210157 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:13.211668 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.211905 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.212404 30988 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:13.213469 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.214357 31691 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
W20260709 12:26:13.214828  4511 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.002s
I20260709 12:26:13.215291  4511 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.216789 31142 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:13.041590  4643 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.217408  4643 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.044886  4607 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.175s
I20260709 12:26:13.220683  4607 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:13.221062  4677 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.222024  4677 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:13.222651 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 161424us
W20260709 12:26:13.224453  4565 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.000s
I20260709 12:26:13.224820  4677 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:13.225063  4565 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.221110  4607 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:13.225310 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.070379  4604 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.156s
I20260709 12:26:13.226444  4628 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:13.226289  4607 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.226819  4628 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:13.226972 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.227622  4607 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:13.227952  4628 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.228506 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:13.229202  4628 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:13.207708 31141 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
W20260709 12:26:13.221122 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.233698 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.235137 31694 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:13.237358 31062 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:13.225582  4565 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.256592 31138 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
W20260709 12:26:13.259079 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.221468  4672 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:13.235332  4527 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:13.262121 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.262434 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.266598 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.251466  4673 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:13.276301 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.285641 31064 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:13.286881  4545 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.287523  4545 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.289569  4545 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:13.290411 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.290665 31061 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:13.293484 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.282040 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.270874  4549 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:13.298059 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.299607 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.306826 31064 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:13.309628 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:13.315922 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.314963 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.319674 30992 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:13.320977  4673 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.323114  4656 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:13.325724 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.327867 31137 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:13.328987 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.338562 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.346724 30987 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:13.315692  4690 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.124s	user 0.000s	sys 0.004s
W20260709 12:26:13.350342 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 83596.3us
W20260709 12:26:13.350962 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.352237 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:13.355160  4674 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:13.359113 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.360201 30992 raft_consensus.cc:2410] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:13.365032 31060 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
W20260709 12:26:13.345561  4693 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.213s	user 0.004s	sys 0.000s
I20260709 12:26:13.353682 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:13.375465  4511 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:13.377652 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.353632  4594 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:13.380278  3591 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:13.381155  4643 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:13.381768  4051 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.605s	user 0.013s	sys 0.000s
I20260709 12:26:13.380300 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.392087 31139 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:13.392493  4647 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:13.399101  4685 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:13.297691  4672 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.409053 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.425923 31060 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:13.430280 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.434927  4382 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.448436  4382 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.450840  4382 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:13.450999  4527 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.451726  4527 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.413018  4674 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:13.434959  4565 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:13.438371  4545 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:13.408697  4463 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:13.446467 30989 raft_consensus.cc:2368] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:13.447077 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.447752 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:13.457329 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 231981us
I20260709 12:26:13.458236 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.466032 31139 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:13.468117 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 114545us
I20260709 12:26:13.473132  4545 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.442701  4612 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:13.474350 31142 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:13.454202  4527 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:13.496516 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.505968 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.476318  4674 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.532436 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2520) took 1417 ms. Trace:
I20260709 12:26:13.532189  4674 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.527134 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:13.507745 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:13.444414  4051 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.510270  4339 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:26:13.514370 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.522634 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:13.527112  4545 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:13.527606 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.529013 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.529214  4672 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:13.530845  4561 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:13.536898 31138 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:13.550858 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.551007 31141 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:13.551529 31138 raft_consensus.cc:2368] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15.
I20260709 12:26:13.552829 30988 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:26:13.536944 31137 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:13.538529 30985 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:13.539247 31139 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:13.548511 30987 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:13.555845 30991 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:13.557550 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 55082.1us
I20260709 12:26:13.559398 30992 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:13.556473 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:13.560966 31690 raft_consensus.cc:2368] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:13.561846 31140 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 14: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:26:13.559540 31135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2505) took 2637 ms (client timeout 2827 ms). Trace:
I20260709 12:26:13.564163 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:13.566092 31690 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:26:13.562582 31135 rpcz_store.cc:269] 0709 12:26:10.922342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.025560 (+103218us) service_pool.cc:224] Handling call
0709 12:26:13.548744 (+2523184us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b5400063454. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:13.559341 (+ 10597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4924364160}
I20260709 12:26:13.567848 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.532855 31144 rpcz_store.cc:276] 0709 12:26:12.114491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.116304 (+  1813us) service_pool.cc:224] Handling call
0709 12:26:12.118987 (+  2683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.598872 (+479885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.598921 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.599658 (+   737us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.917010 (+317352us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:13.177241 (+260231us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.199290 (+ 22049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.199449 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.522300 (+322851us) raft_consensus.cc:1685] finished
0709 12:26:13.522320 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.532232 (+  9912us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27411,"prepare.queue_time_us":186962,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":168593,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":2647,"thread_start_us":1019286,"threads_started":3,"wal-append.queue_time_us":651567}
I20260709 12:26:13.570864 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.571842 31135 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:13.573138 31135 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:13.575722 31142 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:26:13.576753 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.579684 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.580734 31143 raft_consensus.cc:2368] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20.
I20260709 12:26:13.556269  4527 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:13.583885 31138 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:13.550204  4561 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:13.587455 31139 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:26:13.567521  4545 raft_consensus.cc:2764] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:26:13.591877 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.594907 31140 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:13.603679 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 46229.3us
I20260709 12:26:13.607322 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.610661  4339 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:13.610786 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.608662 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.613832 31141 raft_consensus.cc:2368] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13.
I20260709 12:26:13.611390 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.618340 31143 raft_consensus.cc:2368] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19.
I20260709 12:26:13.615608 31139 raft_consensus.cc:2376] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20. Re-sending same reply.
I20260709 12:26:13.621449 30990 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:13.621627 30985 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:13.622537 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.623077 30985 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:13.623791 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:13.624171  4698 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.202s	user 0.000s	sys 0.003s
I20260709 12:26:13.624974 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.625892 30991 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:13.616803  4339 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:13.628280 31142 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:13.629696 30990 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:13.629870 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.630523 30988 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:13.609824  4674 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:13.631052 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.627787 30989 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:13.634459 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.635294 31141 raft_consensus.cc:2368] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19.
I20260709 12:26:13.630532 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:13.638283 31143 raft_consensus.cc:2368] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15.
W20260709 12:26:13.643189 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 357213us
I20260709 12:26:13.628819  4511 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.654548 30992 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:13.648296 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.655172 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.654523 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.633826  4565 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.656353 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.661722 31140 raft_consensus.cc:2368] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17.
W20260709 12:26:13.654624 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 302808us
W20260709 12:26:13.663056 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.656814 31142 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:13.650255 31109 leader_election.cc:336] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.535s (SENT)
I20260709 12:26:13.646284  4527 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:13.662490 31141 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:26:13.669209 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:13.655848  4674 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:13.674790 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.683432 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.684063 30987 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:13.685128 30885 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:13.665421  4565 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:13.703382 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.704763 30989 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
W20260709 12:26:13.675972  4696 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:26:13.711099 31141 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:26:13.712273 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:13.712946 31141 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:13.715948 31139 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:13.717165 31139 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:26:13.718523  4707 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:13.718731 31140 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:13.671052  4511 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:13.722491 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.719044  4707 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.711138 30992 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:13.724226 31139 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:26:13.706707 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:13.676301  4563 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:13.727677 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 108332us
I20260709 12:26:13.680123  4673 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:13.731611 31135 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:13.736647 31142 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:13.732568 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:13.703784  4051 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:13.739570 30989 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:13.742791 31139 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 12:26:13.743994 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 70625.9us
I20260709 12:26:13.682087  4527 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.748132 30987 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:13.748541 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.750385  4628 consensus_meta.cc:287] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.004s
W20260709 12:26:13.751474 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 106525us
I20260709 12:26:13.756557 30988 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:26:13.757880 31141 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:13.760056 30990 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:26:13.747783  4607 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:13.765491 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:13.749491  4543 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:13.767540 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.676616  4339 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.774619 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.760994 30885 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 10 to 12, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 12 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.770678 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.769711 30987 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:26:13.774792 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.705888  4561 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:13.739248  4563 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.783957 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:13.784174  4339 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:13.784739 31690 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:13.787573  4382 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:13.789757  4563 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:13.774159  4677 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:13.801190 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:13.802206 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 126516us
W20260709 12:26:13.803329 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 53866.3us
I20260709 12:26:13.803699  4463 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:13.804077 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:13.804342 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:13.807792  4463 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:13.809491  4463 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:13.809930  4463 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:13.811331  4527 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:13.811643  4594 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:13.811944  4527 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:13.812351  4594 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:13.780822  4511 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:13.800195  4673 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.814719  4673 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.813585  4594 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.816757  4673 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:13.817750 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:13.825126  4654 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:13.829131  4565 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:13.831527 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.820745  4670 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.821283  4672 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:13.821753 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.810977  4463 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.822496 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.856338 31698 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:13.831717 30987 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:13.832172 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:13.833945 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.777993  4628 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:13.834417 30992 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:13.820127  4561 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:13.838701 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.842159 30991 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:26:13.838106 31692 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:13.862156 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.862311 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.862401 31140 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:13.863161 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.861846 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.865197 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.856590 30885 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:13.866685 31697 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:13.872841 31033 leader_election.cc:400] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:13.873183 31033 leader_election.cc:403] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:26:13.875660 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.882812  4719 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:13.883920  4719 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:26:13.922143 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.846601  4677 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:13.925594 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.926816 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:13.791020  4607 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.929826 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:13.930107 31691 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:13.924252 30990 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 12.
I20260709 12:26:13.932324 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.862679  4674 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.930797 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:13.935559 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:13.937656 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 58892.8us
I20260709 12:26:13.937650 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.939797 31690 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:13.945263 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.930368  4463 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:13.945693 30992 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:13.860531  4672 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:13.950837 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 54066.9us
I20260709 12:26:13.932680  4051 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:13.956847 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.958474 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.965415 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:13.966127 31690 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:13.963263 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:13.859700  4670 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.968690 31033 leader_election.cc:400] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
W20260709 12:26:13.947849  4717 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.058s	user 0.000s	sys 0.003s
I20260709 12:26:13.971959 31033 leader_election.cc:403] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:13.972947 31062 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:13.974856 30882 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 321) took 23847 ms (client timeout 15000 ms). Trace:
I20260709 12:26:13.976975  4672 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:13.980697  4707 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:13.991206 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:13.928373  4654 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:13.929534  4628 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:13.987926 31067 raft_consensus.cc:2410] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:26:13.997912 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.998984 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 60350.9us
I20260709 12:26:14.000437 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.000823 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:14.000851 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:14.002530 30988 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:14.002041 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:14.003813 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:14.004614 30991 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:13.942577  4674 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.010303  4703 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:14.010877  4703 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:14.011790  4703 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.014905 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:14.016152  4607 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:14.018353  4628 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:14.020722 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48788.2us
I20260709 12:26:14.022373 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.023253 31698 raft_consensus.cc:2410] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:26:14.023986 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.024303 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.025137 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.025488 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.026587 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.026832 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:14.026998 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.027501 31058 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:14.028028 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.028519 30985 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:14.028955 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.029130 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.032948 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:14.034090 30986 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:14.034689  4677 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.003172  4707 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
W20260709 12:26:13.985769 30882 rpcz_store.cc:269] 0709 12:25:49.910717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.910940 (+   223us) service_pool.cc:224] Handling call
0709 12:26:13.753402 (+23842462us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:04.342382 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:04.343061 (+   679us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:04.343114 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:26:04.343142 (+    28us) tablet.cc:662] Decoding operations
0709 12:26:04.347006 (+  3864us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:04.347091 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:04.347119 (+    28us) tablet.cc:685] Acquiring locks for 31 operations
0709 12:26:04.349332 (+  2213us) tablet.cc:701] Row locks acquired
0709 12:26:04.349381 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:26:04.349537 (+   156us) write_op.cc:270] Start()
0709 12:26:04.349702 (+   165us) write_op.cc:276] Timestamp: P: 1783599964349469 usec, L: 0
0709 12:26:04.349744 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:26:04.451358 (+101614us) log.cc:844] Serialized 12006 byte log entry
0709 12:26:04.777128 (+325770us) op_driver.cc:464] REPLICATION: finished
0709 12:26:05.791476 (+1014348us) write_op.cc:301] APPLY: starting
0709 12:26:05.791574 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:06.689321 (+897747us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:06.724337 (+ 35016us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:07.202120 (+477783us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:07.202608 (+   488us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=62,key_file_lookups=62,delta_file_lookups=62,mrs_lookups=0
0709 12:26:07.202672 (+    64us) write_op.cc:312] APPLY: finished
0709 12:26:07.203221 (+   549us) log.cc:844] Serialized 268 byte log entry
0709 12:26:08.279150 (+1075929us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2ee878a0ac 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b
0709 12:26:09.552420 (+1273270us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:09.567506 (+ 15086us) write_op.cc:454] Released schema lock
0709 12:26:09.582861 (+ 15355us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:09.676016 (+ 93155us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:09.681313 (+  5297us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1290853,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":1014026,"apply.run_cpu_time_us":145425,"apply.run_wall_time_us":3826895,"cfile_cache_hit":126,"cfile_cache_hit_bytes":133769,"num_ops":31,"prepare.queue_time_us":199062,"prepare.run_cpu_time_us":16759,"prepare.run_wall_time_us":279549,"raft.queue_time_us":83518,"raft.run_cpu_time_us":10074,"raft.run_wall_time_us":111558,"replication_time_us":427188,"spinlock_wait_cycles":445319936,"thread_start_us":2661966,"threads_started":5,"wal-append.queue_time_us":2469306,"wal-append.run_cpu_time_us":22533,"wal-append.run_wall_time_us":2202962}]]}
W20260709 12:26:14.043772 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55285.2us
I20260709 12:26:14.044611 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:14.041880 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.045867 31690 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:26:14.042411  4707 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:14.007354 31109 leader_election.cc:336] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.714s (SENT)
I20260709 12:26:14.027673 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:14.049947 31697 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:14.050235 30987 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:14.054445 31138 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 12:26:14.054920 31138 pending_rounds.cc:85] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 11
I20260709 12:26:14.055186 31138 pending_rounds.cc:107] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:26:14.055469 31138 raft_consensus.cc:2889] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:14.056165 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:14.056134 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 49504.8us
I20260709 12:26:14.067418  4051 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:14.068383 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.069068 31694 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
W20260709 12:26:14.050849  4722 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:14.044688  4670 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:14.060102 30885 catalog_manager.cc:5697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:14.082544 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.083959 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:14.085237 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.085850 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:14.058066  4563 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:14.091223 30988 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:14.092785 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:14.095368 31138 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:14.096082 31664 leader_election.cc:400] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15.
I20260709 12:26:14.097226 30987 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:14.098191 30985 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:26:14.098093 31696 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:14.098659 30985 pending_rounds.cc:85] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 13
I20260709 12:26:14.098953 30985 pending_rounds.cc:107] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:26:14.099174 30985 raft_consensus.cc:2889] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:14.071422  4707 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.100948  4707 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:14.104985 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:14.083177  4565 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:14.108950 31059 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:14.105975  4654 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.120915 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:14.121860 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:14.104770  4647 consensus_meta.cc:287] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.659s	user 0.010s	sys 0.001s
W20260709 12:26:14.126332 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 81875.8us
I20260709 12:26:14.113591 30991 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:14.106199 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 103188us
I20260709 12:26:14.131678 30885 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 5 to 8, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:14.132978 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:14.134487 31109 leader_election.cc:336] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.756s (SENT)
I20260709 12:26:14.134989 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:14.135957  4051 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:14.136410 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:14.136724  4674 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:14.127074 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:14.137342 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.137281  4672 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:14.138266  4672 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:14.138793  4670 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:14.139178 30984 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:14.139637  4670 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:14.107334  4565 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:14.140697  4670 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.140985  4565 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.142989 30885 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:14.145236  4529 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.147758 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:14.147684  4529 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.148600 31059 raft_consensus.cc:2410] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:14.137842  4594 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:14.113226  4628 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.150493  4628 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.151103  4463 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:14.152097  4719 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:14.152458  4719 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:14.152586  4628 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:14.145741 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:14.146293  4674 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:14.139212  4672 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.149922  4594 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:14.153390  4719 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.161784  4594 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.163010  4719 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:14.157295  4543 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:14.165973 31068 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:14.168226 31697 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:14.161737 31695 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:26:14.155264 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:14.107003  4725 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:26:14.156869 31664 leader_election.cc:400] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20.
I20260709 12:26:14.105756 30989 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:14.157743  4674 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:14.157298  4635 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:14.173454 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:14.174060 31059 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:14.179183 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:14.179960  4647 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:14.182806 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.184820 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:14.184180 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:14.193325  4607 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:14.154954  4561 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:26:14.194175  4607 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:14.194599  4607 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:14.194811  4561 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:14.180347  4647 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.195257  4561 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:14.195551  4647 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:14.195492  4607 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.197691  4607 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.173419  4674 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:14.196205  4561 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.159988  4529 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:14.205021 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 70606us
W20260709 12:26:14.205793 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48226.5us
W20260709 12:26:14.212061 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.218916  4565 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:14.190331  4543 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:14.228538  4594 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:26:14.229708 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.229960  4594 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:14.228669 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44599.2us
I20260709 12:26:14.229220  4543 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:14.231511 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:14.231976  4543 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.232847  4561 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:14.237877  4725 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:14.239166  4594 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.251636 31144 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:14.255092 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:14.255446 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:14.255708 31140 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:14.256225 31143 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:14.256662 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:14.258569 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.262080 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:14.262683 31694 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:14.264825  4675 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:14.265002 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 108302us
I20260709 12:26:14.265997 31664 leader_election.cc:400] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13.
I20260709 12:26:14.267057  4725 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:14.267525  4725 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:14.267740 31664 leader_election.cc:400] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19.
I20260709 12:26:14.268296  4725 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.269099 31664 leader_election.cc:400] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15.
I20260709 12:26:14.269649  4725 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:14.270133 31664 leader_election.cc:400] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19.
I20260709 12:26:14.270509  4628 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:14.271180  4675 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:14.271040  4628 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.272156  4675 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.273200  4628 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:14.271040 31664 leader_election.cc:400] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17.
I20260709 12:26:14.278697 31138 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:14.278924 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:14.279269 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:14.278712 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:14.279753 31135 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:14.280097 31139 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
W20260709 12:26:14.281890 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.248162  4565 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:14.253428  4463 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:26:14.315367 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:14.316246 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:14.316977 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:14.318982 31059 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:26:14.324131 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 64071us
I20260709 12:26:14.324962 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:14.326179 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.322623  4707 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:14.327441 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:14.328966 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.322935  4670 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:14.360028 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:14.342636 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 71696.7us
I20260709 12:26:14.364447 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:14.365446  4628 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:14.365959 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:14.329505 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 98114.6us
I20260709 12:26:14.324296  4672 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:14.328671  4382 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:14.333053  4673 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:14.326941  4707 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:14.375056 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 58137.2us
I20260709 12:26:14.359794  4051 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:14.377254 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.377131 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:14.366930  4670 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.369963  4628 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:14.370869  4672 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:14.384929 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:14.366982  4527 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:14.378473  4707 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:14.414889  4736 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:26:14.416210  4731 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
W20260709 12:26:14.418496 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 46353.9us
I20260709 12:26:14.385926  4654 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:14.420598  4654 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.421032  4654 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:14.385628  4670 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:14.409024  4677 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:14.433450  4677 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:14.439661 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:14.440742  4563 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:14.440948 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:14.443760  4529 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:14.445232  4703 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:14.445748  4703 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:14.446350 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.446703  4703 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.447160  4677 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.449121  4647 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.451133  4594 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:14.449195 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.451767  4594 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.452251  4594 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:14.452548  4563 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:14.452968  4563 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:14.453330  4563 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:14.410205  4672 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:14.455183 30958 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.458920 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.461678  4635 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:14.444149 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 59295.2us
W20260709 12:26:14.511436 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.384931  4051 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:14.564359 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.564929 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 97051.5us
I20260709 12:26:14.569171 31139 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:14.571448  4607 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:14.574622 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 109902us
I20260709 12:26:14.466400  4529 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:14.538954  4382 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:26:14.606164 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.576418  4628 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:26:14.565665  4051 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:14.626390 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 266403us
I20260709 12:26:14.583351  4339 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:26:14.587294  4635 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:14.649461 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:14.379370  4565 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:14.661813 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 202956us
I20260709 12:26:14.663270  4635 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:14.678900  4635 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:14.686267 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.605180  4607 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.688283  4607 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:14.690058  4674 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.693007  4670 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.667416  4746 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:14.696866  4654 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.590771  4673 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:14.699033  4673 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.678951  4057 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 12:26:14.704362 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:14.707214 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:14.705464  4674 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:14.604547  4529 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:14.688402  4707 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.731343  4527 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:14.732364  4057 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:26:14.739262 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.721061  4607 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.771497  4647 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:14.774102 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 201098us
I20260709 12:26:14.776311 31063 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:14.778210 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:14.719666  4672 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.782464 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:14.781055 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.614s (SENT) [suppressed 48 similar messages]
I20260709 12:26:14.795061  4764 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.783244  4607 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.797392  4561 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:26:14.798385 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.799314  4607 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:14.800695  4654 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:14.808756  4749 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:14.809019  4628 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:14.809196 31140 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:14.809463  4628 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:14.809861 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 87643us
I20260709 12:26:14.812074  4635 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.812594  4753 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:14.795733  4764 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.816215  4764 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:14.813970  4628 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.817395 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:14.815600 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:14.821943 31138 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:14.823124 31139 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:14.824833  4051 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:14.827919 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45571.2us
I20260709 12:26:14.775427  4527 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:14.835523 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.838521 31063 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:14.840404 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:14.814167  4635 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:14.835497 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 235050us
I20260709 12:26:14.847733 31135 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:14.832927  4754 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:14.840440  4527 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:14.853617 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.858479 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.859396 31063 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:14.764675  4382 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.825901  4339 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:26:14.862494 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.852380  4746 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:14.864498 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:14.846762  4763 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:14.875967 31063 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:14.866703 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 217315us
I20260709 12:26:14.878741 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.878909 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.882148 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:14.881819 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:14.862673  4339 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:14.862807  4754 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:26:14.896489 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 81099.6us
I20260709 12:26:14.882948  4628 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:14.892620  4670 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:14.901084  4745 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:14.901821  4755 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:14.903798  4754 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.905364  4757 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.905998  4757 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.906829  4339 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.908129  4339 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:14.909240  4757 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:14.909708 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:14.910095  4757 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:14.884627  4673 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:14.910635  4757 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:14.910789  4673 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.911317  4673 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:14.883917  4767 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.861071  4594 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:14.850945  4529 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:14.918295  4757 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:14.880385  4749 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
W20260709 12:26:14.960660 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.900456  4770 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:14.895445  4563 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:14.990410 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 83454.5us
I20260709 12:26:14.945353  4707 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:14.891484  4750 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:26:14.999482 31063 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:14.998764 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:14.999899 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:15.003623 31696 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:14.984686  4770 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:15.006331 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 80738.8us
I20260709 12:26:15.007303 31144 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:15.009428 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:15.016062 31138 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:15.020687 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:15.021538 31138 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:15.023411 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:15.023979 31141 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:14.926299  4767 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.024183 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:15.033943 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 155271us
W20260709 12:26:15.052313 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 52516us
I20260709 12:26:15.034422 31136 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:15.053227 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:15.015337  4051 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:15.033413  4769 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:26:15.008776  4770 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.061698 30958 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:15.026769  4674 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:15.014642  4755 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:15.023981  4672 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:15.033514  4760 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.110127 31695 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:15.114754 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:15.056064  4753 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:15.034127  4777 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:15.058252  4745 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:15.120775 31141 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:15.062307  4654 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:26:15.122145 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:15.122936 30990 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:15.110724  4670 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:26:15.127305 31060 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:26:15.127770 31060 pending_rounds.cc:85] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 13
I20260709 12:26:15.151899 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:15.151091 31060 pending_rounds.cc:107] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:26:15.155534 31060 raft_consensus.cc:2889] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:15.157750 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45380.3us
I20260709 12:26:15.161700 31694 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:15.164444  4759 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.165165  4759 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.165697  4767 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:15.168546 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:15.124367  4749 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:15.168100 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:15.178707 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 55616.8us
I20260709 12:26:15.177749 31138 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:14.998544  4746 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:15.181268 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 125716us
I20260709 12:26:15.150036  4753 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:15.166914  4769 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:15.186120 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:15.118412  4760 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.124779  4745 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:15.190397 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 155698us
I20260709 12:26:15.187455 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:15.200942 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.204401 31109 leader_election.cc:336] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.799s (SENT)
I20260709 12:26:15.124686  4674 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:15.213408  4674 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.215062  4776 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.215637  4776 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:15.216075  4776 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.179903  4675 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:26:15.175171  4767 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:15.232849 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:15.197069  4760 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.196846  4745 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.202039  4607 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:15.150168  4777 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.238566 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:15.194739  4724 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:26:15.211900  4594 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:15.239373 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 60541.2us
I20260709 12:26:15.232007  4785 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:26:15.245800 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:15.221292  4775 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.228664  4563 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:15.257216 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 47038.4us
I20260709 12:26:15.238838  4607 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:15.261749 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.237809  4672 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:26:15.249361  4057 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:15.276145 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.278270 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.279851 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 55546.7us
I20260709 12:26:15.206724  4755 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.287220 31068 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:15.287842 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:15.288470 31060 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 12.
I20260709 12:26:15.292351  4792 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:15.292940  4792 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:15.293802  4792 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.259848  4775 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:15.298290  4762 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
W20260709 12:26:15.299494  4766 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
W20260709 12:26:15.300428  4765 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:26:15.300695  4777 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.306209 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:15.298254  4775 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:15.310437  4723 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:15.310009  4770 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.311014  4723 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:15.311914  4723 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.314486  4770 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:15.318177 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.322742  4781 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 12:26:15.338832 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 74131.8us
I20260709 12:26:15.351612 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:15.352051 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.282657  4750 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.283519  4774 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.153s
I20260709 12:26:15.287225  4759 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.397190 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:15.398705 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 92579.2us
I20260709 12:26:15.246369  4753 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:15.391878  4789 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:26:15.358029  4674 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:15.354445  4339 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:26:15.395351  4767 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:15.409614 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.412781 31140 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:15.411418 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.415243 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.418076 31135 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
W20260709 12:26:15.418391 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.418974 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:15.416468 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 178026us
I20260709 12:26:15.403143  4759 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.423101 31144 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:15.403095  4672 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.427209 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:15.427379 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:15.425252 31696 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:15.427911 31138 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.425882 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.428527 31137 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:26:15.370527  4673 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.430905 31136 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:15.431553 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.430768 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:15.435007 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:15.436079 31694 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:15.392896  4750 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.439244 31141 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:15.443550 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:15.444144 31697 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:15.430145  4767 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.449100 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:15.450623 31033 leader_election.cc:336] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.270s (SENT)
W20260709 12:26:15.452852 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.453521  4793 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:15.454298  4780 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:15.454077  4793 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.454799  4775 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:15.456262  4339 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:15.456645  4339 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.457252  4339 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.459297  4724 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:15.436910  4382 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:15.461884  4382 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.462829  4724 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:15.464249  4382 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.464761  4724 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:15.466451  4778 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:15.466589  4750 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.403254  4607 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.466616 31143 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:15.467026  4778 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.467573  4607 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.432035  4759 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.469285  4778 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.469807  4778 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.470192  4759 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.470414  4771 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:26:15.471239  4594 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:15.472359  4594 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:15.472703  4594 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:15.473423  4673 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.473455  4594 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.474669  4594 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:15.474074  4777 consensus_queue.cc:1175] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" responder_term: 15 status { last_received { term: 14 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:15.478533 31138 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
W20260709 12:26:15.474027  4761 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.485s	user 0.000s	sys 0.005s
I20260709 12:26:15.480638 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.481177 31138 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:15.482753 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:15.485426  4792 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:15.485965  4792 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:15.432695  4057 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:15.486795 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:15.456568  4793 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:15.493144 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45696.4us
W20260709 12:26:15.494025 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 81616.1us
I20260709 12:26:15.437407  4795 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:15.462111  4527 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.599s	user 0.003s	sys 0.006s
W20260709 12:26:15.491370  4757 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.003s
I20260709 12:26:15.433236  4672 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.517166 31689 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:15.440454  4790 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.111s
I20260709 12:26:15.490686  4057 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:15.540725 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.506522  4673 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.551959 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:15.557838 31697 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:15.489683  4778 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.564280 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:15.541076  4718 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.490281  4767 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.512007  4543 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:15.511881  4760 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:15.578423 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 143480us
I20260709 12:26:15.544162  4057 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:15.572367 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.514866  4527 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.595537 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:15.589877  4795 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:15.604365 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 61534.3us
I20260709 12:26:15.611229 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:15.591560  4628 consensus_meta.cc:287] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.003s
I20260709 12:26:15.591378  4543 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.569509  4725 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:26:15.523955  4757 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.597571  4594 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:15.623974 31137 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 12:26:15.627149 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 51019.7us
I20260709 12:26:15.571641  4673 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.614148  4759 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:15.592631  4718 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.638841 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:15.628324  4543 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:15.640702 31696 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:15.587849  4792 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.645628  4673 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:15.645782  4801 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:15.638988  4764 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:15.647781  4764 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:15.648648  4764 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.649807  4764 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:15.636689  4563 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.627822  4776 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.637923  4594 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:15.662456  4673 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:15.683566 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:15.688055 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 55968.6us
I20260709 12:26:15.684459 31137 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:15.688899 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:15.690455  4795 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.683936 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:15.691543 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:15.697475 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 86335.2us
I20260709 12:26:15.691509 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:15.699446  4382 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.699831  4594 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.706408  4757 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:15.706969 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:15.707276  4815 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:15.707700  4815 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:15.707763  4529 raft_consensus.cc:2764] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:26:15.708146  4815 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:15.709142 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:15.709920  4529 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:15.676549  4628 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:15.650512  4759 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.679765  4755 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:15.721786  4339 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.716259 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:15.684077  4565 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.666630  4563 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:15.724922 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.714331  4754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.001s	sys 0.032s
I20260709 12:26:15.738833 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:15.720710  4755 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.743139 31144 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:15.741384 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:15.723306  4723 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:15.729511  4754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.001s	sys 0.033s
I20260709 12:26:15.745253 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:15.745852 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 54373.5us
I20260709 12:26:15.750910 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:15.746191 31033 leader_election.cc:400] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:15.746955 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:15.680609  4745 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:15.729636  4529 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.758932 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:15.759286 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:15.760977 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.761617 31695 raft_consensus.cc:2368] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20.
I20260709 12:26:15.762109 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.733847  4760 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.748260  4755 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:15.762158 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:15.765702 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:15.748652  4723 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:15.775826 31144 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:15.749339  4778 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:15.780719 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.730865  4565 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.759449  4527 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:15.761494  4673 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:15.806191 31691 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:15.807544 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:15.812687 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:15.761296  4767 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:15.762811  4529 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:15.814774 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:15.819885  4749 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:15.783675  4723 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:15.832804 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:15.833581 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 71531.9us
W20260709 12:26:15.835223 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.836884  4745 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:15.837193  4563 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:15.837831  4718 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:15.806121  4339 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:15.808934  4795 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:15.847813 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:15.831755  4529 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.842799  4565 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:15.859273 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:15.852274 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.863549 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:15.845496  4749 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.825640  4750 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.864140  4382 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:15.862507  4770 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:26:15.848666  4776 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:15.865285  4543 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:15.830852  4796 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:15.839674  4527 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:15.876845 31139 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 19: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:15.865973  4749 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:15.875002  4673 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:15.862113  4057 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:15.931684  4807 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 12:26:15.946727  4820 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:26:15.947832  4817 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 12:26:15.886693  4795 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.004829 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 95588.6us
I20260709 12:26:16.001433  4703 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.005750 30984 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:16.006094  4703 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.006974 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.008288 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.009174 30985 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:16.010859 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.008270  4703 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.014765 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.015604 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.020100 30988 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:16.020156 30983 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:16.012718 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:15.997535  4749 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.016376 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2714) took 1018 ms. Trace:
I20260709 12:26:16.016943 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:16.017891 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.022282 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:16.049912 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:16.081066 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:15.999176  4563 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:16.047883 31143 rpcz_store.cc:276] 0709 12:26:14.997972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.998098 (+   126us) service_pool.cc:224] Handling call
0709 12:26:16.016215 (+1018117us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b540006efd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:16.016322 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013508992}
I20260709 12:26:16.092319 31136 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 12:26:16.108727 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58901.2us
I20260709 12:26:16.109295 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.109938 30992 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:16.108350 30987 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:16.116519 30988 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:16.120404 31143 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:16.121460 31141 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:16.125016 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.125492 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.125890 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.135128 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.125232 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.016111  4759 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:16.024044  4675 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.143229 30985 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:16.142885 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.149564 31060 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.082147  4673 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:16.145720 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.152961 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.108971  4828 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.123685  4750 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:16.014966  4796 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 12:26:16.111708  4749 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.175508 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.002461  4795 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:16.190735  4824 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:26:16.190994 30987 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:16.191244  4755 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.180593 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.181881 31143 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:16.176399 31068 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:16.192369 31697 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:16.192963 30992 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:16.194352  4723 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.196743  4723 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.198827 31060 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:16.119238  4563 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.208258 30985 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:16.215238  4749 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.215591  4755 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:16.217641  4816 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:26:16.218312  4779 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.218827  4779 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.220975  4779 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.237138 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.238703 30983 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:16.179818  4561 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:16.160411  4675 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.250660 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.253888 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:16.256997 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:16.264117  4830 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
W20260709 12:26:16.265724  4831 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:26:16.279498  4563 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:16.281272  4815 consensus_meta.cc:287] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.003s
I20260709 12:26:16.281747  4815 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.283638  4795 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:16.252795  4561 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.286979  4561 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.179186  4828 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.289166  4828 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:16.291460  4826 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.283s	user 0.002s	sys 0.000s
I20260709 12:26:16.295240  4529 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.297801  4672 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.299331 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.299515 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.300005 30985 raft_consensus.cc:2376] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21. Re-sending same reply.
I20260709 12:26:16.301153 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.301895 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.302623 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.303650 30987 raft_consensus.cc:2410] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:26:16.305480  4815 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:16.306033 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.303547 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:16.306469  4780 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.852s	user 0.004s	sys 0.010s
I20260709 12:26:16.306792  4795 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:16.307230  4051 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:16.308070  4675 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.309818 30984 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:16.316421  4057 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.318377 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.318614 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.383827 31060 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:16.322849 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.322575 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.384546 31696 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:16.329324 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.330019 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.330806 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.387758 31058 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:16.388907 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.389827 31058 raft_consensus.cc:2368] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:26:16.331975  4745 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:16.332744 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:16.392167 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 59494.8us
I20260709 12:26:16.393932  4745 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:16.394492  4745 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.394929  4745 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.394956  4707 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:16.395774  4707 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:16.351008 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 97225.3us
I20260709 12:26:16.353021 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:16.377388 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.399201 30983 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:16.399114 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:16.407712 31061 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:16.378599 30992 raft_consensus.cc:2376] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20. Re-sending same reply.
W20260709 12:26:16.408627 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.380810 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:16.381174 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 256285us
I20260709 12:26:16.381548 31689 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:16.383880 31068 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:16.322520 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.398376  4707 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:16.411247 31065 raft_consensus.cc:2410] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:16.411688 31697 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:16.409813 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:16.412534 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59619.9us
I20260709 12:26:16.412632 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.414484 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:16.416906  4051 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.417402 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:16.418043  4838 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
I20260709 12:26:16.419905  4057 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:16.297801  4529 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:16.375320  4785 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.425942 31697 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:16.430189 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.380509  4771 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:16.381673  4780 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:16.381191  4757 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.439406 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.383319  4795 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.440315 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.442668 31143 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:16.443382 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.443819  4795 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:16.444505 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.445693 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2721) took 1014 ms. Trace:
I20260709 12:26:16.446514 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:16.446707 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:16.447247 31696 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:16.449950 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.446038 31135 rpcz_store.cc:276] 0709 12:26:15.431665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.480832 (+ 49167us) service_pool.cc:224] Handling call
0709 12:26:16.002459 (+521627us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:16.008630 (+  6171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.008665 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:16.012419 (+  3754us) log.cc:844] Serialized 52 byte log entry
0709 12:26:16.154880 (+142461us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:16.154907 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.154951 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.330934 (+175983us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:16.334395 (+  3461us) raft_consensus.cc:1685] finished
0709 12:26:16.334417 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.445534 (+111117us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54003f4dd0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:16.445639 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":629,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":3422,"spinlock_wait_cycles":1260599936,"thread_start_us":74086,"threads_started":1,"wal-append.queue_time_us":157111}
I20260709 12:26:16.450590 31144 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:16.424455  4815 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:16.463910  4051 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:16.434744  4707 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:26:16.435385  4771 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.465449  4707 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:16.465893  4707 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:16.466980 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:16.466795  4707 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.467963  4707 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:16.439072  4785 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.470508  4785 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.451999  4563 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:16.471868 31062 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:26:16.385119  4543 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:16.474020 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.474908 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.435289 30983 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:16.421361  4777 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.453752 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.476722 31135 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:16.454636 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:16.456243 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:16.450024  4757 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.461679  4795 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:16.432085  4057 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:16.465757  4771 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.496580 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.510325 31064 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:26:16.514851 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.516409 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.517066 31137 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:16.514484 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:16.532356 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 76206.3us
I20260709 12:26:16.533703 31141 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.518301 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:16.534947 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.542371 31136 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:16.531533 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.539634 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:16.544090 31135 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:16.544827 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.554036 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.474581  4755 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:16.510474  4051 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:16.559939 31106 leader_election.cc:400] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:26:16.560282 31106 leader_election.cc:403] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:16.556802 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.564958 31062 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:16.567232  4563 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.567757  4563 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.452764  4779 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 12:26:16.569136  4779 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:16.569475  4779 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:16.570282  4779 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.571940  4779 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:16.558458  4755 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:16.576865  4755 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.577929 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.579025 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.579568 31143 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:16.580304 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.581282 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.582341 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.585350 31139 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:16.570019  4563 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.587669 31143 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:16.588366  4057 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.596513  4745 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.589028 31143 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:26:16.598660 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56035.6us
I20260709 12:26:16.600646 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:16.558139 31068 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:16.602200 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:16.603811 30956 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.082s	sys 0.113s
W20260709 12:26:16.604135 30956 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.082s	sys 0.113s
I20260709 12:26:16.605872 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.607326 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.607820 30983 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:16.609017 31141 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:16.609514 30992 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:26:16.609992 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57603.9us
I20260709 12:26:16.614921 31058 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:16.615656 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.618324 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.619453 31065 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:16.554639 31135 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:16.622246 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.613257 31144 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:16.625311 31141 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:16.555699  4543 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:16.625991 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:16.521879  4718 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.094s
I20260709 12:26:16.627375 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:16.623468 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:16.510838  4777 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.632284 30983 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:16.636343 31135 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:16.638533 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.639158 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:16.615669  4795 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16.
I20260709 12:26:16.652799 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.658704 30983 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:16.652984 30990 raft_consensus.cc:2410] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:16.654047 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.659368 30987 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:26:16.556931  4770 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:26:16.660394 31141 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:16.669798 31135 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:16.672616 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:16.682514 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:16.674907 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:16.629701  4785 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:26:16.691728 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.692343 31065 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.694694 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.629814  4746 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:26:16.623189  4527 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:26:16.638566  4543 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:16.698876  4543 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:16.699926  4777 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.668532  4745 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:16.683229  4757 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:16.702536  4777 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:16.704545  4771 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.655035  4654 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:16.706655  4654 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:16.707513  4654 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.707921  4771 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:16.650967  4780 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:26:16.710948  4780 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:16.711336  4780 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:16.712164  4780 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.713385  4780 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:16.640792  4755 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.714746  4673 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.715389  4673 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.716375  4755 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.717839  4673 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:16.651290  4759 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:16.723726 31697 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:16.724368 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:16.745169 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.745829 30983 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:16.672003  4818 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.765532 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.723265  4776 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:16.766839 31062 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.765874 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:16.770466 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.777105 31065 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 13.
I20260709 12:26:16.778275 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.779619 30983 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:16.779137 30985 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:26:16.779923 31062 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:16.781867 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:16.782050 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.741765  4672 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:16.784359 31661 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:16.786134  4795 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:16.794395  4776 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:16.794910  4776 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:16.795740  4776 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.797578  4051 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:16.798563 31142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.016s	sys 0.008s
I20260709 12:26:16.799427 30987 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:16.800205 30990 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:16.820520 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:16.830505  4543 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:16.768517  4707 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
W20260709 12:26:16.842723 31138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.013s	sys 0.029s
W20260709 12:26:16.843588 31138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.013s	sys 0.029s
I20260709 12:26:16.833153  4543 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.847453 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:16.779016  4818 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:16.849138 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 431201us
W20260709 12:26:16.798975 31142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.017s	sys 0.008s
I20260709 12:26:16.857090 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.857784 30990 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:16.862389 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1167) took 1441 ms. Trace:
I20260709 12:26:16.835026  4795 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:16.871483 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:16.879675 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.866889  4828 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:26:16.880548 30990 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:16.880172  4795 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:16.881124 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.855072  4672 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:16.876971  4785 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.886607  4672 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:16.887539  4672 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:16.866720  4754 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:16.892679  4818 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:16.894641 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.895815 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.895897 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.896633 30990 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:16.896947 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.897477 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.898536 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.899134 30991 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:26:16.899595 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.900069 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.901110 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.866469  4563 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:16.902117 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.898559 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.897953 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:16.919054 30983 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.898128 30990 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:16.860291  4565 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 12:26:16.944085 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.950865 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 103503us
I20260709 12:26:16.951694 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:16.953732 30986 raft_consensus.cc:2393] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 10: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:26:16.868003  4647 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.005s	sys 0.042s
I20260709 12:26:16.962817 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:16.969866  4647 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.006s	sys 0.042s
I20260709 12:26:16.862879 31142 rpcz_store.cc:276] 0709 12:26:15.421090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.421379 (+   289us) service_pool.cc:224] Handling call
0709 12:26:15.432372 (+ 10993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:15.432570 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:15.440815 (+  8245us) log.cc:844] Serialized 14 byte log entry
0709 12:26:16.279121 (+838306us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b18004b0100. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:26:16.861883 (+582762us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:16.861954 (+    71us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:16.862038 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.862177 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.862324 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5649,"prepare.queue_time_us":8123,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"spinlock_wait_cycles":5429120,"thread_start_us":576846,"threads_started":1}
W20260709 12:26:16.952883 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.014s	sys 0.014s
W20260709 12:26:16.972040 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.015s	sys 0.015s
W20260709 12:26:16.972332 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 551554us
W20260709 12:26:16.973541 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 58247.7us
I20260709 12:26:16.979475 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:16.974435 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:16.982793  4707 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:16.983628 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:16.985755 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:16.956521  4828 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:17.011907 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.014635 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:17.015497 30990 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:16.995626 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:17.018194 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:17.019573 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 67976.8us
I20260709 12:26:17.022353 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:17.023950 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:17.025027 30983 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
W20260709 12:26:17.029327 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 55005.9us
W20260709 12:26:17.033195 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48959.2us
W20260709 12:26:17.033784 31693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.014s	sys 0.013s
W20260709 12:26:17.034181 31693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.014s	sys 0.013s
I20260709 12:26:17.030272 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:17.035873 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:17.024645  4792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.020s	sys 0.036s
I20260709 12:26:17.021126  4843 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.045161 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:17.033650  4527 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:17.072535  4527 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:17.073460  4527 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:17.074406  4563 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:17.075389  4563 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:17.058946  4673 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:17.035740  4746 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.043056  4707 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:26:17.042851  4815 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:17.062736 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2729) took 1549 ms. Trace:
W20260709 12:26:17.086138 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 411281us
I20260709 12:26:17.046823  4382 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:17.050770  4759 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:17.065168  4792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.020s	sys 0.036s
I20260709 12:26:17.035570  4750 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.064163 30882 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 20, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 20 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:17.071053  4703 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:16.993786  4051 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:26:17.076406  4842 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:17.078151 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:17.084698 31064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.012s	sys 0.009s
I20260709 12:26:17.035902  4770 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:17.083554 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:17.085237 31138 rpcz_store.cc:276] 0709 12:26:15.513415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.701404 (+187989us) service_pool.cc:224] Handling call
0709 12:26:15.701758 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:15.702121 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.702153 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:16.324888 (+622735us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b2c000e06f8. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:26:16.325434 (+   546us) log.cc:844] Serialized 52 byte log entry
0709 12:26:16.844550 (+519116us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:16.877504 (+ 32954us) log.cc:844] Serialized 14 byte log entry
0709 12:26:16.877696 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.877761 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.883259 (+  5498us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b1800285700. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:17.062528 (+179269us) raft_consensus.cc:1685] finished
0709 12:26:17.062551 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.062680 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33652,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":12809,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":15470,"spinlock_wait_cycles":37739264,"thread_start_us":518723,"threads_started":1,"wal-append.queue_time_us":734950}
I20260709 12:26:17.093726 30882 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 15, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 15 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:17.097052 31064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.012s	sys 0.009s
I20260709 12:26:17.098345 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:17.089043  4815 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:17.107626 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:17.102910 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:17.109635 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:17.101055 31059 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:17.091699  4856 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.090337  4673 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:17.104497 30882 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:17.114185 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:17.112353 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:17.097177  4703 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.097757  4749 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:17.101126  4529 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:17.118153 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:17.116608 30958 leader_election.cc:400] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20.
I20260709 12:26:17.111644  4815 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:17.125332 30885 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 314) took 59199 ms (client timeout 15000 ms). Trace:
I20260709 12:26:17.112919  4675 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:17.096630  4750 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:17.116999 31140 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.015s	sys 0.007s
I20260709 12:26:17.097635  4770 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.097594  4842 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:17.124822 30958 leader_election.cc:403] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:26:17.116882  4828 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:17.117830  4753 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.116539  4790 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.123085  4703 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:17.119910  4839 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:17.116652  4853 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:17.135784 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:17.122931  4749 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:17.122010  4856 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.143036  4755 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.138973  4780 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:17.138440 31140 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.015s	sys 0.007s
I20260709 12:26:17.115170 30882 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 7 to 9, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 9 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.133649  4529 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:17.139950  4770 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.142349  4828 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.151088  4753 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20.
I20260709 12:26:17.191898 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2862) took 2075 ms. Trace:
I20260709 12:26:17.195067 30882 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:17.155005  4839 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.186375  4818 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:17.186326 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 50597.6us
I20260709 12:26:17.202193 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:17.197405 30882 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 19, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 19 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:17.207899 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 92100us
I20260709 12:26:17.189642  4779 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:17.131603 30885 rpcz_store.cc:269] 0709 12:25:17.924039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.924232 (+   193us) service_pool.cc:224] Handling call
0709 12:26:17.123467 (+59199235us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:15.018699 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:15.254415 (+235716us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:15.263081 (+  8666us) write_op.cc:435] Acquired schema lock
0709 12:26:15.264596 (+  1515us) tablet.cc:662] Decoding operations
0709 12:26:15.347188 (+ 82592us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:15.347308 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:15.347349 (+    41us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:26:15.348195 (+   846us) tablet.cc:701] Row locks acquired
0709 12:26:15.348251 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:26:15.348439 (+   188us) write_op.cc:270] Start()
0709 12:26:15.348606 (+   167us) write_op.cc:276] Timestamp: P: 1783599975348371 usec, L: 0
0709 12:26:15.348650 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 12:26:15.406099 (+ 57449us) log.cc:844] Serialized 2501 byte log entry
0709 12:26:15.838715 (+432616us) op_driver.cc:464] REPLICATION: finished
0709 12:26:16.221752 (+383037us) write_op.cc:301] APPLY: starting
0709 12:26:16.221891 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:16.310010 (+ 88119us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:16.310127 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:16.313793 (+  3666us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:16.314089 (+   296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 12:26:16.314144 (+    55us) write_op.cc:312] APPLY: finished
0709 12:26:16.314469 (+   325us) log.cc:844] Serialized 65 byte log entry
0709 12:26:16.324535 (+ 10066us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:16.324645 (+   110us) write_op.cc:454] Released schema lock
0709 12:26:16.324706 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:16.327092 (+  2386us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:16.327120 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":152157,"reported_tablets":47,"spinlock_wait_cycles":78080,"child_traces":[["op",{"apply.queue_time_us":382677,"apply.run_cpu_time_us":9986,"apply.run_wall_time_us":105359,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29219,"num_ops":6,"prepare.queue_time_us":229360,"prepare.run_cpu_time_us":20570,"prepare.run_wall_time_us":628836,"raft.queue_time_us":144616,"raft.run_cpu_time_us":84938,"raft.run_wall_time_us":295523,"replication_time_us":489860,"thread_start_us":607443,"threads_started":4,"wal-append.queue_time_us":259049}]]}
I20260709 12:26:17.196753  4828 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:17.192745  4843 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.196467 30989 rpcz_store.cc:276] 0709 12:26:15.117017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.117141 (+   124us) service_pool.cc:224] Handling call
0709 12:26:15.123708 (+  6567us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:15.124105 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.124136 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:15.164407 (+ 40271us) log.cc:844] Serialized 52 byte log entry
0709 12:26:16.376107 (+1211700us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b1800291ca0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:26:16.783097 (+406990us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:17.141991 (+358894us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.152742 (+ 10751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.152814 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.152852 (+    38us) raft_consensus.cc:1685] finished
0709 12:26:17.191098 (+ 38246us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.191780 (+   682us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10301,"prepare.queue_time_us":276307,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":43392,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":34506,"spinlock_wait_cycles":6551424,"thread_start_us":758530,"threads_started":2,"wal-append.queue_time_us":1871715}
I20260709 12:26:17.218732 30882 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.154719  4790 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.221019  4856 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:17.222183 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:17.242506  4675 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:17.242432  4853 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:26:17.151051  4842 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:17.247957  4750 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:17.187772  4852 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:26:17.248950 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:17.256836  4815 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.197224  4780 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:17.202632  4839 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.193849  4755 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.187258  4673 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:17.259773 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 46804.8us
I20260709 12:26:17.261296 30882 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 20, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 20 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:17.276414 30882 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 17, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 17 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:17.279253 30882 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 20, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 20 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:17.260102  4782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.018s	sys 0.036s
I20260709 12:26:17.217474  4057 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:26:17.261991 31067 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.018s	sys 0.036s
I20260709 12:26:17.258250  4529 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
W20260709 12:26:17.288712 31067 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.018s	sys 0.036s
I20260709 12:26:17.293212  4770 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:17.284883 30882 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:17.300688 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:17.287395  4782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.018s	sys 0.037s
I20260709 12:26:17.297966 30882 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 17, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 17 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:17.305240  4771 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.305882  4771 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.308455  4771 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:17.285673  4852 consensus_queue.cc:1243] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:17.312904  4749 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:17.314795 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:17.314651  4780 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.278867  4750 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.318756 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:17.318717  4750 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.319808  4770 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:17.321390  4750 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:17.276801  4853 consensus_queue.cc:1243] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:17.321995 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:17.286319  4842 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:17.323796  4850 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:17.320724  4672 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:17.324204  4850 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:17.324472  4672 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:17.325081  4850 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.304411 30882 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.326468  4850 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.327407  4563 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:17.327790  4563 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:17.327133 30882 catalog_manager.cc:5697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 8 to 16, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 16 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:17.328541  4563 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.214329  4818 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.272287  4779 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:17.266163  4843 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:17.344161 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 194458us
W20260709 12:26:17.409143 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 363992us
W20260709 12:26:17.669864 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.670646 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.669952 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 351261us
W20260709 12:26:17.561600 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 345051us
I20260709 12:26:17.673295  4753 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:17.691326 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:17.410140 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 377658us
I20260709 12:26:17.694983 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:17.560981 31061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.014s	sys 0.020s
W20260709 12:26:17.696141 31061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.014s	sys 0.023s
I20260709 12:26:17.688941 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3030) took 1270 ms. Trace:
W20260709 12:26:17.561450  4776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.000s	sys 0.022s
W20260709 12:26:17.560477 30939 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.000s	sys 0.018s
W20260709 12:26:17.675513 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.669142 30882 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:17.706701 30882 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 13 to 20, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 20 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:17.708254 30882 catalog_manager.cc:5697] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 16. New cstate: current_term: 16 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.690804 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:17.683888  4672 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.710009  4672 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:17.710007 30882 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.711747 30882 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:17.716739 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:17.717046 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:17.717324 31140 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:17.717569 31138 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.722347 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:17.722625 30882 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:17.723837 30882 catalog_manager.cc:5697] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:17.725013 30882 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:17.726441 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:17.727202 30983 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.678473  4780 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.674283  4771 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:17.698349  4776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.000s	sys 0.025s
I20260709 12:26:17.709136 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:17.754632 30983 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:17.732273 31696 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:26:17.768594 31696 pending_rounds.cc:85] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 11
I20260709 12:26:17.769493 31696 pending_rounds.cc:107] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:26:17.770752 31696 raft_consensus.cc:2889] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:17.692780  4749 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:17.773070 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:17.744688 31138 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:17.764467 31697 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:17.696614 31064 rpcz_store.cc:276] 0709 12:26:16.402066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.521217 (+119151us) service_pool.cc:224] Handling call
0709 12:26:16.521480 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.521696 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:16.554976 (+ 33280us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.408491 (+853515us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:17.672286 (+263795us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:17.672338 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.672404 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.672526 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97964,"prepare.queue_time_us":33397,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"thread_start_us":755123,"threads_started":2,"wal-append.queue_time_us":754765}
I20260709 12:26:17.765362 31692 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:17.773262 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:17.776239 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:17.776932 31135 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:17.779930 31143 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:17.774057 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:17.774534 31693 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:17.769136 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:17.698010  4863 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:17.818944 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 109825us
I20260709 12:26:17.698140  4754 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.746402 30882 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 13, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 13 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:17.821913 30882 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 17, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 17 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.816684 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3036) took 1090 ms. Trace:
I20260709 12:26:17.825146 30882 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 13 to 17, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 17 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:17.789350  4647 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.789814 31139 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:17.790310 31137 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:17.790791 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:17.790894 31140 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:17.789855 31141 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:17.794884  4779 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.795782 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:17.797428  4815 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:17.798619  4755 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:17.801343  4703 consensus_meta.cc:287] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.614s	user 0.015s	sys 0.006s
W20260709 12:26:17.806463  4839 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.530s	user 0.000s	sys 0.015s
I20260709 12:26:17.806952  4563 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.811762  4750 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.764293  4057 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:17.764459  4852 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.859141 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:17.797950 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2865) took 1192 ms. Trace:
I20260709 12:26:17.764293  4051 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:26:17.745899  4529 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:17.815603  4828 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.003s
W20260709 12:26:17.727905 30939 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.001s	sys 0.021s
I20260709 12:26:17.824168 31061 rpcz_store.cc:276] 0709 12:26:16.726682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.726943 (+   261us) service_pool.cc:224] Handling call
0709 12:26:16.742455 (+ 15512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.742789 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.742816 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.743206 (+   390us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.696618 (+953412us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:17.696657 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.696718 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.816485 (+119767us) raft_consensus.cc:1685] finished
0709 12:26:17.816505 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.816631 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3977,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":12112,"thread_start_us":913974,"threads_started":1,"wal-append.queue_time_us":979039}
I20260709 12:26:17.713551  4675 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.110s
I20260709 12:26:17.842862 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:17.843374 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:17.843678 30987 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:17.844480 30992 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: [14.10-16.11]
I20260709 12:26:17.833537 30882 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.853183 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:17.820369  4759 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.810084  4707 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:17.856189 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 157084us
I20260709 12:26:17.788163  4780 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.858165 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:17.866587 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:17.864719 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:17.860103 30984 rpcz_store.cc:276] 0709 12:26:16.605156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.605281 (+   125us) service_pool.cc:224] Handling call
0709 12:26:16.618440 (+ 13159us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:16.626493 (+  8053us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.626521 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:16.640361 (+ 13840us) log.cc:844] Serialized 78 byte log entry
0709 12:26:17.103054 (+462693us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:17.103108 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.103193 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.797721 (+694528us) raft_consensus.cc:1685] finished
0709 12:26:17.797746 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.797893 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3634,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":18963,"spinlock_wait_cycles":6528,"thread_start_us":449294,"threads_started":1,"wal-append.queue_time_us":1064685}
I20260709 12:26:17.854557  4853 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.857390  4850 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:17.888016 30989 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:26:17.886914 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 193742us
I20260709 12:26:17.897255  4675 consensus_queue.cc:1243] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:17.891160 30983 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:17.894343 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 199428us
I20260709 12:26:17.835410  4647 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:17.879133  4828 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:17.926453 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 73372.4us
W20260709 12:26:17.963635 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 107884us
I20260709 12:26:17.963857 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:17.900544 30882 catalog_manager.cc:5697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 6 to 16, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 16 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.928930 31068 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:17.855515  4839 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:17.850625  4703 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:17.868139  4815 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:26:17.869603  4770 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:26:17.854768  4745 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:26:17.830579  4754 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:17.890610 31064 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:17.869778  4750 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:17.956669 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:18.004498 30986 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:17.924727 31059 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:17.897595  4790 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:26:17.925345  4875 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:17.887123  4852 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:26:17.964452 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:17.979559 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:17.944818  4759 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:17.906816  4707 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.015218 30990 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:17.953758  4529 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:26:17.982200 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:17.986863  4843 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:17.991223  4871 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:17.991990 31139 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:17.940052  4057 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:17.994525 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:18.003983  4565 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:18.011231 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 47448.8us
I20260709 12:26:17.981640  4839 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:18.016361 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59758.4us
I20260709 12:26:17.958599  4673 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:18.019340 31141 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:17.962203  4782 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:18.021314 31144 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
W20260709 12:26:18.019064  4842 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.006s
I20260709 12:26:18.001921 30882 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:17.958881  4850 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.019999  4790 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.008975  4881 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:18.019851  4795 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:18.018464  4759 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:18.032068 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:18.044524 30882 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:17.955255  4853 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.023670  4843 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:18.049872 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:18.051970  4859 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:26:18.060812  4780 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.052569  4749 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
W20260709 12:26:18.062044  4864 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:26:18.062597 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:18.062958 30991 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:18.073271 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:18.074002  4818 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:18.075261  4870 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:17.973992  4885 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:26:18.076211  4885 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:18.076565  4885 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:18.077389  4885 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.070348 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 943) took 1627 ms. Trace:
I20260709 12:26:18.079349  4703 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.078446 31689 rpcz_store.cc:276] 0709 12:26:16.435711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.436472 (+   761us) service_pool.cc:224] Handling call
0709 12:26:16.436765 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.849790 (+413025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.849836 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.869224 (+ 19388us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.155501 (+286277us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:17.411601 (+256100us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.674314 (+262713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.674382 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.063065 (+388683us) raft_consensus.cc:1685] finished
0709 12:26:18.063086 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.063246 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12073,"raft.queue_time_us":19358,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":24157,"thread_start_us":805972,"threads_started":3,"wal-append.queue_time_us":958041}
I20260709 12:26:18.018148  4875 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.082145  4828 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:18.082501  4875 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:18.083345  4749 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:18.084021  4750 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:18.049994  4881 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:18.085210  4749 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:18.085669  4749 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:18.049616  4884 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:18.086618  4884 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.088716  4884 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.088902  4885 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.089960  4754 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:18.095497 30988 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:18.111150 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.111785 30983 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:18.113019 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.113703 30983 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:18.085405  4881 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.063382  4755 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:18.241197 30986 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:18.261142 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:18.263999 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.264214 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.266150 31137 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:26:18.263283 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 273499us
I20260709 12:26:18.026791  4057 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.050632  4759 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:18.259294 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.067785  4839 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.264951 30989 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:18.071642  4863 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:18.272428 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:18.272541 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:18.269659 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:18.276516 31068 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:18.256532  4776 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.279737 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3031) took 1877 ms. Trace:
I20260709 12:26:18.269528  4780 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.043469  4673 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.265475 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:18.269698  4818 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:18.269585  4856 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.067968  4843 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.067553  4782 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:18.255864 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 226523us
I20260709 12:26:18.324970 30986 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:18.312211 31106 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:18.281252 31067 rpcz_store.cc:276] 0709 12:26:16.402453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.523278 (+120825us) service_pool.cc:224] Handling call
0709 12:26:16.523610 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.539250 (+ 15640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.539279 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.547711 (+  8432us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.289157 (+741446us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:17.289212 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.289300 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.279548 (+990248us) raft_consensus.cc:1685] finished
0709 12:26:18.279565 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.279682 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17124,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":14213,"thread_start_us":662323,"threads_started":1,"wal-append.queue_time_us":1213777}
W20260709 12:26:18.324889 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 56388us
W20260709 12:26:18.305730 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 341356us
I20260709 12:26:18.329329 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:18.276325  4863 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:18.347276 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:18.348042 30983 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:18.351393 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.352584 30987 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:18.292121  4881 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.372714 31141 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:18.332773  4529 consensus_queue.cc:1243] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:18.436009  4529 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:26:18.449079 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 121990us
I20260709 12:26:18.453437 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:18.460259 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:18.476316 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 144371us
I20260709 12:26:18.341418  4883 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.357738  4863 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:18.275938  4749 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.485754 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:18.308406  4780 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:18.510775 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 50610.7us
I20260709 12:26:18.291203  4776 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.527245 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:18.530685  4675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.025s	sys 0.083s
I20260709 12:26:18.310941  4818 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.527463 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:18.450815  4918 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:18.487496  4899 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 12:26:18.486318  4867 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.367s	user 0.000s	sys 0.001s
W20260709 12:26:18.536917 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 206442us
I20260709 12:26:18.312209  4856 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.544958 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:18.532055 31690 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:18.560461 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.615114 30990 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:18.561219 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:18.566061 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:18.544034  4675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.026s	sys 0.084s
I20260709 12:26:18.495023  4897 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:18.551559 30983 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:18.613586 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:18.558036  4707 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.617061  4755 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:18.616165 31693 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
W20260709 12:26:18.666031 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 52744.5us
I20260709 12:26:18.559130  4918 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.672439 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:18.542608  4750 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:26:18.551720  4853 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:18.573465  4749 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:18.643359 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 306539us
I20260709 12:26:18.567126  4850 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.524304  4883 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.686691  4883 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.566644  4703 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.104s
I20260709 12:26:18.686842 31689 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:18.645814  4852 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:18.662477  4543 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:18.709637  4926 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:18.723228 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3035) took 1997 ms. Trace:
I20260709 12:26:18.677834  4755 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.710260  4672 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.724128  4903 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.724182  4672 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.724781  4903 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.726900  4903 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.678273  4918 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:18.743054 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 226912us
W20260709 12:26:18.708837  4563 consensus_meta.cc:287] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.850s	user 0.007s	sys 0.003s
I20260709 12:26:18.687314  4883 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:18.691850  4853 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.723650 31058 rpcz_store.cc:276] 0709 12:26:16.726333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.738511 (+ 12178us) service_pool.cc:224] Handling call
0709 12:26:16.746394 (+  7883us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.747446 (+  1052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.747501 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.747993 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.867601 (+119608us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4800210258. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef07bdaa1 00007f2ef07b4b37 00007f2ef07b49ca 00007f2ef08ac26f 00007f2ef08abf37 00007f2ef08c055c 00007f2ef08c0508 00007f2ef08c0488 00007f2ef08c0443 00007f2ef08bf6ec 00007f2eef7e9a96
0709 12:26:17.208874 (+341273us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:17.208921 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.208986 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.711124 (+1502138us) raft_consensus.cc:1685] finished
0709 12:26:18.711151 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.723161 (+ 12010us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":638,"raft.queue_time_us":767,"raft.run_cpu_time_us":6095,"raft.run_wall_time_us":113749,"spinlock_wait_cycles":24609024,"thread_start_us":415289,"threads_started":1,"wal-append.queue_time_us":1250032}
I20260709 12:26:18.707518  4057 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.771572 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:18.711604  4543 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:18.779755 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:18.751299  4790 consensus_meta.cc:287] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.701s	user 0.009s	sys 0.006s
W20260709 12:26:18.762313  4779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.012s	sys 0.048s
W20260709 12:26:18.791706  4779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.012s	sys 0.048s
I20260709 12:26:18.727869  4673 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:18.774950 31695 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:18.757056  4903 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:18.662125  4897 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.723492  4909 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:26:18.767992  4850 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:18.787799  4543 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:18.792383  4790 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.787160  4920 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:18.764730  4675 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 12:26:18.832502 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 160130us
I20260709 12:26:18.801608  4057 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:18.782316  4883 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.795120  4921 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.815493  4843 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:18.807015  4673 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:18.802407  4927 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:18.798982  4853 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.834899  4920 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.824297  4892 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:18.847512 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:18.828758  4779 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:18.856021 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:18.828070  4707 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:18.837102  4563 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:18.882436 31696 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:18.841151  4057 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:18.892596 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 365222us
I20260709 12:26:18.874984  4843 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.871053  4780 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.882365  4920 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:18.883937  4818 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.924024 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.883544  4707 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:18.871971  4921 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.935971  4856 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:18.937213  4921 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.888275  4779 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.939198  4779 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:18.897857  4057 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.928973 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2642) took 1083 ms. Trace:
I20260709 12:26:18.929662 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.946012 30990 raft_consensus.cc:2376] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21. Re-sending same reply.
I20260709 12:26:18.947342 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.948175 30990 raft_consensus.cc:2376] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19. Re-sending same reply.
I20260709 12:26:18.931529 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2639) took 1089 ms. Trace:
I20260709 12:26:18.933215 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2846) took 1005 ms. Trace:
I20260709 12:26:18.949056 30988 rpcz_store.cc:276] 0709 12:26:17.842512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.842624 (+   112us) service_pool.cc:224] Handling call
0709 12:26:18.931372 (+1088748us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400226d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:18.931470 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372694784}
I20260709 12:26:18.872634  4887 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:26:18.894001  4892 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:18.874630  4673 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.945830 30991 rpcz_store.cc:276] 0709 12:26:17.845186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.845314 (+   128us) service_pool.cc:224] Handling call
0709 12:26:18.928819 (+1083505us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540022d8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:18.928919 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348183424}
I20260709 12:26:18.958807 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.959511 30988 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:18.960723 30988 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:18.954661 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:18.949122 31063 rpcz_store.cc:276] 0709 12:26:17.927685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.927793 (+   108us) service_pool.cc:224] Handling call
0709 12:26:17.947807 (+ 20014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.029843 (+ 82036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.029884 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:18.029938 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:18.029986 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.932949 (+902963us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b54002ce7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:18.933009 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.933149 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188087552}
W20260709 12:26:18.970252 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 188486us
I20260709 12:26:18.898159  4563 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.996369 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:18.940999  4707 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:18.947160  4750 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.956179  4901 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 12:26:19.009722 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:19.010571 30991 raft_consensus.cc:2368] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:18.912847  4843 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:18.996037  4915 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.922015  4927 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 12:26:19.022954  4853 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:19.024887  4868 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 12:26:19.026355  4904 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 12:26:19.029265  4885 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.940s	user 0.008s	sys 0.000s
W20260709 12:26:19.029567  4906 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.730s	user 0.003s	sys 0.000s
I20260709 12:26:19.004526  4896 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 12:26:19.030876  4928 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:19.031100  4883 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:19.032351  4935 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:26:19.034863  4879 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:26:19.037402  4920 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.039472 31696 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:19.079648 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.042505  4892 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:18.998805  4924 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.109697 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:19.039897  4647 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.057677 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:19.074476 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.079454 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:19.079674 31690 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:19.082049 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:19.103515 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 107252us
I20260709 12:26:19.045080  4875 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.071520  4759 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.788s	user 0.007s	sys 0.004s
I20260709 12:26:19.105597 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:19.103366 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:19.108774 31696 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:19.012244  4750 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.115157 31140 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:19.126601 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:19.127341 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.128029 31140 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:19.115258 31136 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:19.067114  4921 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.074765  4850 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.067287  4818 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.106827  4883 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.143990 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.135027 31697 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:19.144752 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:19.147655 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.147969 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:19.148765 30986 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
W20260709 12:26:19.135424 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 77876.7us
I20260709 12:26:19.075482  4903 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.124109  4887 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:19.150484 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.149711 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:19.151368 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:19.135934  4795 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.082s	user 0.008s	sys 0.003s
W20260709 12:26:19.153669 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 54704.6us
I20260709 12:26:19.059430  4915 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.165695 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:19.143222  4862 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.143591  4936 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.162802 31030 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.000s	sys 0.029s
W20260709 12:26:19.166680 31030 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.000s	sys 0.029s
I20260709 12:26:19.134564  4897 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:19.167649 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.171657 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2568) took 1392 ms. Trace:
I20260709 12:26:19.172228 31143 rpcz_store.cc:276] 0709 12:26:17.779213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.814514 (+ 35301us) service_pool.cc:224] Handling call
0709 12:26:17.814891 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.832241 (+ 17350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.832275 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.832757 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.030813 (+198056us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:18.030872 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.030955 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.171350 (+1140395us) raft_consensus.cc:1685] finished
0709 12:26:19.171382 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.171582 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16950,"raft.queue_time_us":23077,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":8473,"thread_start_us":154085,"threads_started":1,"wal-append.queue_time_us":954282}
I20260709 12:26:19.175331 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.177986 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2566) took 1399 ms. Trace:
I20260709 12:26:19.178947 31138 rpcz_store.cc:276] 0709 12:26:17.778670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.781409 (+  2739us) service_pool.cc:224] Handling call
0709 12:26:17.781696 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.822201 (+ 40505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.822510 (+   309us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.857679 (+ 35169us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.961978 (+104299us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:17.962031 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.962106 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.029014 (+1066908us) spinlock_profiling.cc:243] Waited 51.3 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:19.161219 (+132205us) raft_consensus.cc:1685] finished
0709 12:26:19.161243 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.177925 (+ 16682us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39178,"raft.queue_time_us":39675,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8398,"spinlock_wait_cycles":117935872,"thread_start_us":63953,"threads_started":1,"wal-append.queue_time_us":352986}
I20260709 12:26:19.181660 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.183377 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:19.182720 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 73113.8us
I20260709 12:26:19.135723  4759 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.161254  4529 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:19.123483  4875 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.187083 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:19.187819 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:19.143919  4921 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.194083 31136 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:19.147241  4850 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.171785  4795 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:19.119351  4924 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.208151 31136 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:19.184620  4647 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.150534  4883 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.151974  4818 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.187511  4749 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:19.188439  4565 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:19.226198 31144 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:19.161562  4903 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.204905  4940 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:19.207242  4902 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:19.233366 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:19.230449 31137 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
W20260709 12:26:19.233608 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 83042.3us
I20260709 12:26:19.212028  4529 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:19.243237 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:19.245286 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:19.246806 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:19.247958 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:19.248842 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:19.230244  4929 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:26:19.247413 31067 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
W20260709 12:26:19.262611 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 62954.1us
I20260709 12:26:19.178423  4862 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.263868 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:19.264180 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:19.266113 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:19.227540  4896 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:19.267881 31062 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:19.268944  4951 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:26:19.237701  4902 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.269109  4923 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.249734  4563 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.271168  4883 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.181710  4936 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.290298  4936 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:19.291013  4936 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:19.234488  4940 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.292668  4940 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:19.233901  4673 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.299226  4673 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:19.301384 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.224401  4647 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.301030  4887 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:19.282013  4863 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.751s	user 0.009s	sys 0.002s
I20260709 12:26:19.280791  4902 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:19.303395  4795 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:19.287892  4923 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.353704 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:19.354606 30987 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:19.326381  4920 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:19.312307  4896 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:26:19.343519  4945 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.370397 30987 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:19.371737  4944 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:19.349309  4750 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:19.341715  4795 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.379436  4924 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.381237  4782 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.425767 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.427732 31138 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:19.357292  4862 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.321794  4863 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:19.443471 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 275199us
I20260709 12:26:19.426452  4795 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:19.435945  4753 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.349694  4529 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:19.478740 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 213815us
I20260709 12:26:19.382903  4887 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.426482  4945 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:19.508566 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 244456us
I20260709 12:26:19.509784 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:19.475431 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 238604us
I20260709 12:26:19.476969  4785 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.477654  4961 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:19.478066  4949 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:19.478899 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:19.446785  4759 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:19.511214 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:19.451519  4915 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:19.591369 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 81666.8us
I20260709 12:26:19.640686 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:19.472016  4850 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.456784  4921 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.482142  4862 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:19.483354  4922 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.670070 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1301) took 1929 ms. Trace:
I20260709 12:26:19.497232  4750 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.486395  4815 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:19.490563  4672 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.491743  4647 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.484800  4707 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.706413 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.707036 31139 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:19.707672  4707 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.707865 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.708840  4960 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:19.709388  4960 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.682566 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.710438 31136 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:19.711573  4960 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:19.666949 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 155843us
I20260709 12:26:19.715819 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:19.657321 31661 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:19.482168  4937 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:19.491137  4923 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:19.493213  4828 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:19.670719 31691 rpcz_store.cc:276] 0709 12:26:17.741076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.741612 (+   536us) service_pool.cc:224] Handling call
0709 12:26:17.775444 (+ 33832us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.948514 (+173070us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.948547 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.949023 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.293801 (+344778us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:18.293944 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.659035 (+1365091us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b5400512a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:19.659094 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.659145 (+    51us) raft_consensus.cc:1685] finished
0709 12:26:19.659162 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.670004 (+ 10842us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":8217,"spinlock_wait_cycles":397102464,"thread_start_us":271971,"threads_started":1,"wal-append.queue_time_us":1320078}
I20260709 12:26:19.495647  4753 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.676707 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:19.679188  4918 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.909s	user 0.009s	sys 0.003s
W20260709 12:26:19.679077 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 200263us
W20260709 12:26:19.679543  4900 log.cc:927] Time spent T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
W20260709 12:26:19.679687  4057 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.715s	user 0.006s	sys 0.005s
I20260709 12:26:19.665268  4862 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.680171  4943 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:19.682062  4856 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:19.682541  4921 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:19.764248  4921 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.480562  4896 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.187s
I20260709 12:26:19.764629  4921 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:19.717133  4843 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.686134  4973 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.694583  4932 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
W20260709 12:26:19.695274  4948 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.385s	user 0.001s	sys 0.000s
I20260709 12:26:19.590070  4933 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.418s
I20260709 12:26:19.703745  4863 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.654517  4944 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.785691  4944 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.786867 31139 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:19.788030  4944 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.674142  4916 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.705233  4938 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:19.788878  4938 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:19.788910  4916 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.789938  4938 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.791024  4938 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:19.792508  4753 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.713050  4750 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.661813  4940 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.794611  4940 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.794708  4750 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.796898  4942 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:19.797434  4942 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:19.800506  4945 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:19.724156  4795 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.803182 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2887) took 1479 ms. Trace:
I20260709 12:26:19.803565 30984 rpcz_store.cc:276] 0709 12:26:18.323550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.323699 (+   149us) service_pool.cc:224] Handling call
0709 12:26:18.371427 (+ 47728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.371687 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.371723 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.447188 (+ 75465us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.751074 (+303886us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:18.751111 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.751177 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.802958 (+1051781us) raft_consensus.cc:1685] finished
0709 12:26:19.802982 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.803124 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":214977,"threads_started":1,"wal-append.queue_time_us":686394}
I20260709 12:26:19.805356 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:19.808739  4755 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.705602  4877 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:19.812634  4877 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:19.813128  4877 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:19.590651  4818 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:19.816929 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:19.817186 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:19.817924 31698 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:26:19.818770  4903 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.819254  4903 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.819650  4903 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.820135 31698 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:19.676749  4815 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:19.821125  4815 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.821798 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:19.724264  4881 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.278s	user 0.011s	sys 0.003s
I20260709 12:26:19.641302  4785 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.763351  4852 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.752823  4828 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.685539 30983 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:26:19.832679 30983 pending_rounds.cc:85] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 10
I20260709 12:26:19.833051 30983 pending_rounds.cc:107] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:26:19.832876 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.833427 30983 raft_consensus.cc:2889] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.822152  4815 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:19.843499  4905 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.115s	user 0.000s	sys 0.002s
I20260709 12:26:19.845769 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2567) took 2066 ms. Trace:
I20260709 12:26:19.846944 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:19.833707 31143 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:19.850279 30990 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:19.846194 31135 rpcz_store.cc:276] 0709 12:26:17.778944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.783609 (+  4665us) service_pool.cc:224] Handling call
0709 12:26:17.783941 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.828536 (+ 44595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.828567 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.850616 (+ 22049us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.958473 (+107857us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:17.958551 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.970457 (+ 11906us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.842759 (+1872302us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:19.845388 (+  2629us) raft_consensus.cc:1685] finished
0709 12:26:19.845416 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.845700 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37485,"raft.queue_time_us":47795,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":15439,"spinlock_wait_cycles":11443584,"thread_start_us":64327,"threads_started":1,"wal-append.queue_time_us":764870}
I20260709 12:26:19.856032 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.856904 31135 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:19.858191 30983 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:19.858758 30983 pending_rounds.cc:85] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 7
I20260709 12:26:19.859062 30983 pending_rounds.cc:107] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:19.859388 30983 raft_consensus.cc:2889] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.769164  4943 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:19.754992  4543 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.920s	user 0.008s	sys 0.004s
I20260709 12:26:19.762383 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:19.866919  4051 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:19.867372  4051 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:19.867857  4051 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.759868  4862 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:19.757332  4672 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.760052  4057 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.677838  4922 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.731439  4937 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.783887  4902 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:19.724381 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 67167us
I20260709 12:26:19.821559  4903 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.766059 31138 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:19.901244 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:19.762756 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:19.865170 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.865597 30983 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:19.904050 30983 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:19.866815 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 104523us
I20260709 12:26:19.753921  4960 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.848244  4828 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.868932 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.925230 31137 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:19.870448 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.926390 31135 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
W20260709 12:26:19.824239  4936 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 12:26:19.864156 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:19.757923  4918 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.888414 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:19.910249  4057 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:19.928107 30984 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:19.911582 31067 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:19.929494 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:19.872803  4943 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:19.930809 31065 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:19.937144 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:19.857900  4852 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.874302  4543 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:19.848897  4973 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.832170  4942 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.005868 31692 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 12:26:20.085117 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 328769us
I20260709 12:26:19.979339  4051 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:19.937665 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:20.101365 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:20.101609 31060 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:19.937150 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:20.103927 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.104283 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.104930 31033 leader_election.cc:400] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:20.105237 31033 leader_election.cc:403] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:20.105166 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.105172 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.106689 31690 raft_consensus.cc:2410] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:20.026358  4903 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.107399  4938 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:20.107937 31697 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:26:20.108332 31697 pending_rounds.cc:85] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 11
I20260709 12:26:20.108539 31697 pending_rounds.cc:107] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:26:20.108750 31697 raft_consensus.cc:2889] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:20.076380 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:20.109760 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.098693 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 954) took 1020 ms. Trace:
I20260709 12:26:20.025616 30988 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:26:20.110608 30988 pending_rounds.cc:85] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 11
I20260709 12:26:20.110996 30988 pending_rounds.cc:107] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:26:20.111471 30988 raft_consensus.cc:2889] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:19.937445 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 73373.1us
I20260709 12:26:20.110173 30983 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:20.110548 31694 rpcz_store.cc:276] 0709 12:26:19.078310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.078416 (+   106us) service_pool.cc:224] Handling call
0709 12:26:19.113722 (+ 35306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.113920 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:19.555698 (+441778us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.098410 (+542712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.098455 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:20.098503 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.098560 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.098648 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54312,"prepare.queue_time_us":167049,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":54959,"thread_start_us":929399,"threads_started":2}
I20260709 12:26:20.108985  4969 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:20.027038  4912 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:26:20.026032  4947 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.843s	user 0.003s	sys 0.003s
I20260709 12:26:20.132510  4051 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.133895 31696 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:20.111151  4926 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.376s	user 0.011s	sys 0.011s
I20260709 12:26:20.128181 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:19.924804  4885 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:20.129096 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:19.994839  4894 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:26:19.938751  4936 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:19.994302  4930 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.091333  4785 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.930446  4795 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.998322  4980 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:20.251705 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2579) took 1182 ms. Trace:
I20260709 12:26:20.252094 31144 rpcz_store.cc:276] 0709 12:26:19.069046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.122824 (+ 53778us) service_pool.cc:224] Handling call
0709 12:26:20.251526 (+1128702us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b54000be5d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:20.251621 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180946304}
I20260709 12:26:20.189807 31696 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:20.071134  4750 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:20.026769  4902 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.025620  4922 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:19.887607  4672 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.191452 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:19.985260  4828 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:20.195755 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:20.119151  4933 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:20.196609 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.124598  4707 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:20.196195 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 77105us
I20260709 12:26:20.203899 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.200989 31689 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:20.126402  4969 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:20.210772 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:19.982873  4818 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.255s
I20260709 12:26:20.198661 31698 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
W20260709 12:26:20.200794  4926 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.474s	user 0.011s	sys 0.012s
I20260709 12:26:20.213922 30983 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:20.119272  4896 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:26:20.218897 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.314580  4896 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:20.315001  4896 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:20.315188 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.315879  4896 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.317041 31690 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:20.124504  4961 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:26:20.125010  4903 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.247454  4930 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.323397  4903 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.324141  4930 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.199798  4771 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.249153  4937 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.325048  4771 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.326086  4852 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.287048  4978 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:26:20.295244  4902 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:20.327507  4902 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.327567  4771 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.250029  4962 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:20.328366  4962 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:20.328956  4852 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.329267  4962 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.250938  4980 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.259395  4718 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:20.284940 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:20.287053 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.291473 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:20.286469 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90805.3us
I20260709 12:26:20.333788 31140 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 18: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:20.261262 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.296268 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.334865 31144 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:20.332144 31690 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:20.259974 31141 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 1187) took 2137 ms (client timeout 2107 ms). Trace:
I20260709 12:26:20.335951  4962 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.336423  4973 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.337273 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:20.337967  4886 negotiation.cc:336] Failed RPC negotiation. Trace:
0709 12:26:17.192324 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.29.37.65:34803)
0709 12:26:18.059675 (+867351us) server_negotiation.cc:207] Beginning negotiation
0709 12:26:18.059712 (+    37us) server_negotiation.cc:400] Waiting for connection header
0709 12:26:19.008569 (+948857us) server_negotiation.cc:408] Connection header received
0709 12:26:19.025583 (+ 17014us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0709 12:26:19.025621 (+    38us) server_negotiation.cc:462] Received NEGOTIATE request from client
0709 12:26:19.025945 (+   324us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0709 12:26:19.026152 (+   207us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0709 12:26:19.331206 (+305054us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0709 12:26:19.333762 (+  2556us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
0709 12:26:19.882080 (+548318us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
0709 12:26:20.192353 (+310273us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
0709 12:26:20.224237 (+ 31884us) server_negotiation.cc:1036] Waiting for connection context
0709 12:26:20.289284 (+ 65047us) negotiation.cc:326] Negotiation complete: Timed out: Server connection negotiation failed: server connection from 127.0.0.1:57992: received 0 of 4 requested bytes
Metrics: {"server-negotiator.queue_time_us":837925,"thread_start_us":348845,"threads_started":1}
I20260709 12:26:20.338531 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.339401 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1178) took 3442 ms. Trace:
I20260709 12:26:20.304064 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:20.335757 31141 rpcz_store.cc:269] 0709 12:26:18.109071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.109259 (+   188us) service_pool.cc:224] Handling call
0709 12:26:20.246457 (+2137198us) spinlock_profiling.cc:243] Waited 81 ms on lock 0x7b540012cfd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:20.246595 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186359040}
I20260709 12:26:20.340750 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:20.213277  4971 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 12:26:20.304426 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:20.334161 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:20.308169 30876 connection.cc:573] client connection to 127.29.37.65:34803 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:26:20.308095  4863 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:20.283835  4933 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:20.309003  4969 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.327965  4902 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:20.329469  4852 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.300920  4918 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.283598 30989 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:20.306922 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:26:20.366920 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60111.7us
I20260709 12:26:20.279592  4952 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.339569 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:20.347038  4947 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:20.332120  4828 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:20.344179 30876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 12 similar messages]
W20260709 12:26:20.283380 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 72697.5us
I20260709 12:26:20.339865 30985 rpcz_store.cc:276] 0709 12:26:16.896717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.900360 (+  3643us) service_pool.cc:224] Handling call
0709 12:26:16.900606 (+   246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.978772 (+ 78166us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:16.993191 (+ 14419us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b1800438760. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf 00007f2eeb8ae038
0709 12:26:17.669647 (+676456us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.860360 (+190713us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.861830 (+  1470us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.862121 (+   291us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:17.862386 (+   265us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.991458 (+129072us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b1800438760. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287
0709 12:26:17.991591 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.991686 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.026976 (+1035290us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b1800438760. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a7522 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:19.302637 (+275661us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b1800438760. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a7522 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:19.871051 (+568414us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b1800438760. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:20.339181 (+468130us) raft_consensus.cc:1685] finished
0709 12:26:20.339202 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.339323 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":263444,"prepare.queue_time_us":82096,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":79,"raft.queue_time_us":241122,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":22541,"spinlock_wait_cycles":46552704,"thread_start_us":595540,"threads_started":3,"wal-append.queue_time_us":253064}
I20260709 12:26:20.339520  4980 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.283140  4750 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.331326  4926 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:20.342846  4949 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:20.317214  4896 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:20.344214 30958 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:20.404188 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 60036.5us
I20260709 12:26:20.375088  4771 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:20.330751  4887 consensus_queue.cc:1175] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a517aadb0ae04b89a13779d7898dfa0c" responder_term: 15 status { last_received { term: 15 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 15 index: 11 } }
I20260709 12:26:20.375202 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.368541 30983 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:20.392292 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.392292 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:20.353237  4933 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:20.352844  4876 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:20.394255 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:20.396131 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.396131 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.999s	user 0.054s	sys 0.097s
I20260709 12:26:20.391481  4969 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:20.347632  4718 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.373584 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.420560 31141 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:20.427016 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.055s	sys 0.098s
I20260709 12:26:20.427225 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:20.428100 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:20.445225 31140 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:20.428699 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:20.430351  4876 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:20.428100 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.431643  4918 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:20.431833  4672 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:20.433055  4862 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.000s
I20260709 12:26:20.434723  4951 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:20.421262  4980 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:20.421150  4771 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:20.437503 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:20.399091  4949 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:20.442248 30989 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
W20260709 12:26:20.449060 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.476622 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1306) took 1622 ms. Trace:
I20260709 12:26:20.476820 30989 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:20.449441  4745 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:20.446959  4565 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.450068  4853 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.426198  4937 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:20.502399 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:20.374965  4952 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.460021 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.464500 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:20.467057  4995 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:26:20.477905 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.447458  4543 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.447062  4818 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:26:20.456195  4918 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.507189 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.508433 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:20.514688 30989 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:20.515601 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:20.477012 31693 rpcz_store.cc:276] 0709 12:26:18.853969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.854135 (+   166us) service_pool.cc:224] Handling call
0709 12:26:18.873191 (+ 19056us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:18.879666 (+  6475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.879710 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:18.892767 (+ 13057us) log.cc:844] Serialized 52 byte log entry
0709 12:26:19.013030 (+120263us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:19.109012 (+ 95982us) log.cc:844] Serialized 14 byte log entry
0709 12:26:19.109427 (+   415us) log.cc:844] Serialized 14 byte log entry
0709 12:26:19.109581 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.109652 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.837033 (+727381us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b18003b0e00. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:20.025309 (+188276us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:20.476418 (+451109us) raft_consensus.cc:1685] finished
0709 12:26:20.476440 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.476558 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":451694,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":92977,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":14744,"spinlock_wait_cycles":33131264,"thread_start_us":205468,"threads_started":2,"wal-append.queue_time_us":226573}
I20260709 12:26:20.516285 30989 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:20.516992 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:20.518579 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.508948 31141 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:20.470083  4980 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.474536  4771 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:20.478670  4863 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:20.489923  4992 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:26:20.490423  4745 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:20.505782  4565 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.534235  4992 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:20.534701  4992 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:20.534513  4745 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.535588  4992 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.536213  4565 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.509531  4991 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:20.475132  4862 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.537002  4991 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.537421  4991 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:20.539036  4862 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.474794  4828 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:20.541908  4877 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.728s	user 0.010s	sys 0.000s
I20260709 12:26:20.493641  4902 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.491150  4916 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.520460  4918 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:20.552019 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.531579 31140 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:20.527281  4980 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:20.487987  4718 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.529598 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:20.511722  4960 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.585s	user 0.005s	sys 0.005s
W20260709 12:26:20.520153  4921 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.755s	user 0.008s	sys 0.002s
I20260709 12:26:20.547379  4902 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:20.558919 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 307146us
I20260709 12:26:20.531780 30989 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:20.554034 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.495842  4943 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.490546  4988 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:26:20.507025  4853 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.557443 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:20.549423  4818 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:20.564440 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.492026  4876 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.557547 30983 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 18: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:26:20.575179 30876 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 231008us
I20260709 12:26:20.491052  4949 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.566521 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:20.587144 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:20.496685  4926 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.561950  4863 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:20.551679  4877 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.600099  4862 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:20.604121 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:20.604228 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:20.605628  4875 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.608201 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2577) took 1539 ms. Trace:
I20260709 12:26:20.580140  4818 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:20.614563 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.623703 31142 rpcz_store.cc:276] 0709 12:26:19.068525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.068657 (+   132us) service_pool.cc:224] Handling call
0709 12:26:19.104008 (+ 35351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.295518 (+1191510us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b1800288ac0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb88cbfb 00007f2eeb89764b
0709 12:26:20.295646 (+   128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.295671 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.296137 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.550628 (+254491us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:20.550680 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.550768 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.607880 (+ 57112us) raft_consensus.cc:1685] finished
0709 12:26:20.607903 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.608136 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21379,"raft.queue_time_us":123055,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":24703,"spinlock_wait_cycles":14868736,"thread_start_us":387362,"threads_started":2,"wal-append.queue_time_us":295850}
I20260709 12:26:20.627264 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.610330  4790 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:20.630664 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.630752 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:20.631282 31142 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:20.631568 31140 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:20.632167 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.632416 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:20.633211 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:20.633525 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.634176 31140 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:20.634980 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:20.627548  4910 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:20.635514  4877 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.637176 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:20.636330  4910 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.636655  5008 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:20.638898  5008 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:20.639302  4943 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.639793 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.640887 31142 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:20.642454 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.643923  4910 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.628782  4949 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:20.641073  5008 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.644862  4952 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.635462 31140 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:20.647141  4776 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:20.560169  4960 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:20.612718 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1196) took 1488 ms. Trace:
I20260709 12:26:20.650746 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:20.625123 31138 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:20.651446 30989 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:20.619771  4862 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:20.653965 31140 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:20.597021  4852 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.616086 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:20.562760  4921 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.661554 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.662670  4921 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:20.663071 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.663420 31696 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:20.664417 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.665174 31694 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:20.665752 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.574834  4672 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:26:20.669049 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.670485 30988 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:20.671818 31693 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:20.600804  4885 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:26:20.628295  4818 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:20.580476  4543 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:26:20.618417  4863 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:20.650625 30991 rpcz_store.cc:276] 0709 12:26:19.123933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.497190 (+373257us) service_pool.cc:224] Handling call
0709 12:26:19.497501 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.848630 (+351129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.848657 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:19.848695 (+    38us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:19.848725 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.612485 (+763760us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:20.612550 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.612669 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807155584}
I20260709 12:26:20.654958 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.654137  4862 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:20.674268 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:20.678580 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.629758 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.684511 31140 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:20.686259 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.687083 30989 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:20.687978 30989 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:20.688748 30989 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:20.691823 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:20.692250 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.693039 31689 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:20.694098 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.707564 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:20.708217 30991 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:20.708657 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:20.675446  4750 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:20.644842 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:20.674818  4921 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:20.695500  5003 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:26:20.686178  4863 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:20.750993 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.770607 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:20.782601 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 185209us
I20260709 12:26:20.828007 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:20.830582 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:20.831318 31064 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:20.836890 31142 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:20.825730  4707 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:20.827106 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:20.872968 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45982.2us
I20260709 12:26:20.891592 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:20.777732  4828 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:20.899806 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 436955us
I20260709 12:26:20.836024  4563 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.902850 31142 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19. Ops: [17.9-18.10]
I20260709 12:26:20.903103 31063 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:20.905625 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:20.906344 31033 leader_election.cc:336] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.710s (SENT)
I20260709 12:26:20.902189 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:20.892892  5003 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:20.907563 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.911700 31696 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:26:20.914197 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:20.924768 31140 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:20.931231 31140 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:20.776708  4921 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.891594  4902 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:20.852336  4790 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:20.878257  4931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.719s	user 0.009s	sys 0.013s
I20260709 12:26:20.940544  4877 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:20.940725  4942 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:20.908936  4563 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:20.937920  4910 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:20.901552  4828 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:21.051250 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1308) took 2183 ms. Trace:
W20260709 12:26:21.320735 31695 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1310) took 2632 ms (client timeout 2139 ms). Trace:
W20260709 12:26:21.759315 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 852939us
W20260709 12:26:21.719910 31064 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.888s	user 0.009s	sys 0.001s
I20260709 12:26:21.760696 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1643) took 1327 ms. Trace:
W20260709 12:26:21.524982 31065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.005s	sys 0.003s
W20260709 12:26:21.042522 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 1205) took 1272 ms (client timeout 1620 ms). Trace:
W20260709 12:26:21.528311 31694 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.004s	sys 0.008s
W20260709 12:26:21.098472 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.525511 30876 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.007s	sys 0.033s
W20260709 12:26:21.827687 30989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.012s	sys 0.035s
W20260709 12:26:21.834174 30989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.012s	sys 0.035s
I20260709 12:26:21.828482 31692 rpcz_store.cc:276] 0709 12:26:18.867776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.867962 (+   186us) service_pool.cc:224] Handling call
0709 12:26:18.868371 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.100001 (+1231630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.050961 (+950960us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54005a5854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:21.051167 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2615299328}
I20260709 12:26:21.835714 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:21.847177 31067 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:21.830173 31064 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:21.843935 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:21.831836 31065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.005s	sys 0.003s
I20260709 12:26:21.853847 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 3059) took 1013 ms. Trace:
W20260709 12:26:21.832865 31694 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.005s	sys 0.008s
W20260709 12:26:21.833426 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.833621 30876 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.008s	sys 0.033s
I20260709 12:26:21.827265 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2765) took 1621 ms. Trace:
W20260709 12:26:21.832317 30984 rpcz_store.cc:269] 0709 12:26:19.769952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.805079 (+ 35127us) service_pool.cc:224] Handling call
0709 12:26:21.042288 (+1237209us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400312c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:21.042445 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280939904}
W20260709 12:26:21.829207 31695 rpcz_store.cc:269] 0709 12:26:18.526491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.528031 (+  1540us) service_pool.cc:224] Handling call
0709 12:26:21.158341 (+2630310us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54002f99d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:21.158494 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363547008}
I20260709 12:26:21.859495 31140 rpcz_store.cc:276] 0709 12:26:20.206070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.019221 (+813151us) service_pool.cc:224] Handling call
0709 12:26:21.826642 (+807421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.826862 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.826898 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.826946 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:21.826984 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.827059 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.827194 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23296}
I20260709 12:26:21.854422 31059 rpcz_store.cc:276] 0709 12:26:20.840422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.843625 (+1003203us) service_pool.cc:224] Handling call
0709 12:26:21.853779 (+ 10154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.873127 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:21.874606 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 3057) took 1022 ms. Trace:
I20260709 12:26:21.875141 31064 rpcz_store.cc:276] 0709 12:26:20.827578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.830262 (+  2684us) service_pool.cc:224] Handling call
0709 12:26:21.849919 (+1019657us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.876221 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:21.877694 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:21.033380  4952 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:26:21.525197  4876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.028s	sys 0.032s
W20260709 12:26:21.053433  4999 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.577s	user 0.001s	sys 0.000s
W20260709 12:26:21.138391  4991 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.003s
I20260709 12:26:20.942786  4902 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:21.646878  4962 consensus_meta.cc:287] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.310s	user 0.010s	sys 0.001s
W20260709 12:26:21.039003  5001 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.643s	user 0.001s	sys 0.000s
W20260709 12:26:21.035943  4997 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.587s	user 0.002s	sys 0.000s
W20260709 12:26:21.526382  4771 consensus_meta.cc:287] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.977s	user 0.011s	sys 0.000s
W20260709 12:26:21.500777  4896 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.084s	user 0.010s	sys 0.000s
W20260709 12:26:21.697527  4980 consensus_meta.cc:287] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.135s	user 0.003s	sys 0.007s
W20260709 12:26:21.049525  4969 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.000s
W20260709 12:26:21.647584  5019 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 12:26:20.945873  4931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.795s	user 0.009s	sys 0.013s
I20260709 12:26:21.830974 31062 rpcz_store.cc:276] 0709 12:26:20.432962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.433106 (+   144us) service_pool.cc:224] Handling call
0709 12:26:20.433430 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:20.439639 (+  6209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.439670 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:20.441863 (+  2193us) log.cc:844] Serialized 52 byte log entry
0709 12:26:20.699435 (+257572us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:20.699471 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.699538 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.671007 (+971469us) raft_consensus.cc:1685] finished
0709 12:26:21.671034 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.760637 (+ 89603us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2813,"raft.queue_time_us":6002,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":2820,"thread_start_us":195488,"threads_started":1,"wal-append.queue_time_us":1193127}
I20260709 12:26:21.882871 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2767) took 1595 ms. Trace:
I20260709 12:26:21.933928 31144 rpcz_store.cc:276] 0709 12:26:20.287020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.829323 (+1542303us) service_pool.cc:224] Handling call
0709 12:26:21.882250 (+ 52927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.882472 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.882509 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.882563 (+    54us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:21.882609 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.882678 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.882804 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.883665 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:21.644102  4862 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.934s	user 0.008s	sys 0.000s
W20260709 12:26:21.031715  4941 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 2.006s	user 0.000s	sys 0.002s
W20260709 12:26:21.649343  4933 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.221s	user 0.001s	sys 0.010s
W20260709 12:26:21.681435  4949 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.036s	user 0.005s	sys 0.005s
I20260709 12:26:21.884845 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 3061) took 1007 ms. Trace:
I20260709 12:26:22.026446 31067 rpcz_store.cc:276] 0709 12:26:20.877795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.872860 (+995065us) service_pool.cc:224] Handling call
0709 12:26:21.884767 (+ 11907us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.027659 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:21.886039 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.028352 31144 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:22.028972 31059 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:21.886859 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2769) took 1599 ms. Trace:
I20260709 12:26:22.029417 31140 rpcz_store.cc:276] 0709 12:26:20.287673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.881369 (+1593696us) service_pool.cc:224] Handling call
0709 12:26:21.886279 (+  4910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.886456 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.886491 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.886532 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:21.886585 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.886660 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.886790 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:21.749985  4918 consensus_meta.cc:287] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.192s	user 0.011s	sys 0.000s
I20260709 12:26:20.952487  4910 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.050495  4937 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:21.048530  4989 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.653s	user 0.002s	sys 0.000s
I20260709 12:26:22.034979 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 3062) took 1135 ms. Trace:
I20260709 12:26:22.035471 31059 rpcz_store.cc:276] 0709 12:26:20.900016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.885762 (+985746us) service_pool.cc:224] Handling call
0709 12:26:22.034891 (+149129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:20.952092  4563 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:21.052493  5004 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:21.889185 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:21.889642 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 3060) took 1017 ms. Trace:
I20260709 12:26:22.042171 31064 rpcz_store.cc:276] 0709 12:26:20.872571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.877409 (+1004838us) service_pool.cc:224] Handling call
0709 12:26:21.889584 (+ 12175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.078575  4853 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:21.890971  4876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.041s	sys 0.036s
I20260709 12:26:21.898608  4991 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:21.895898  4972 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:21.897462  4902 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:21.901616 30991 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.009s	sys 0.017s
W20260709 12:26:22.050450 30991 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.211s	user 0.009s	sys 0.017s
I20260709 12:26:22.050177 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:20.947642  4942 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:21.889855  4952 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:21.912801 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.947188  4877 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:21.077041  4790 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:21.972059 31697 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:21.928721  4980 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.029193 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2770) took 1741 ms. Trace:
I20260709 12:26:22.043001 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.043947 31140 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:21.935770  5009 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.047520 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:22.126003 31144 rpcz_store.cc:276] 0709 12:26:20.287950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.935053 (+1647103us) service_pool.cc:224] Handling call
0709 12:26:22.027941 (+ 92888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.028879 (+   938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.029127 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.127214 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2772) took 1838 ms. Trace:
I20260709 12:26:22.049695 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2599) took 1574 ms. Trace:
I20260709 12:26:22.047673 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2676) took 1221 ms. Trace:
I20260709 12:26:22.127597 31140 rpcz_store.cc:276] 0709 12:26:20.288492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.036511 (+1748019us) service_pool.cc:224] Handling call
0709 12:26:22.043702 (+  7191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.126960 (+ 83258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.127164 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.034502  4918 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:21.914121  4771 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.067797 31067 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:22.033629  4673 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.035341  4910 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.033015  4921 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.023316  4862 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.056739  4972 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.129499 31697 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 1327) took 1299 ms. Trace:
I20260709 12:26:22.042328  4937 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.031095  4933 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.127826 31139 rpcz_store.cc:276] 0709 12:26:20.475056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.048728 (+1573672us) service_pool.cc:224] Handling call
0709 12:26:22.049060 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.049271 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.049307 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.049364 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:22.049431 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.049500 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.049641 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.130339 31062 raft_consensus.cc:2393] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:22.130314 31144 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:22.046835  5004 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:22.135272 31689 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:22.138963 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3063) took 1231 ms. Trace:
I20260709 12:26:22.235529 31067 rpcz_store.cc:276] 0709 12:26:20.907241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.051560 (+1144319us) service_pool.cc:224] Handling call
0709 12:26:22.051890 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.138653 (+ 86763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.138898 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.126302  4896 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.129737  4949 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.215996 30985 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:22.128304 30984 rpcz_store.cc:276] 0709 12:26:20.826644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.872536 (+1045892us) service_pool.cc:224] Handling call
0709 12:26:22.047598 (+175062us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:22.254377 30958 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.343s	user 0.032s	sys 0.021s
I20260709 12:26:22.263054 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1201) took 1770 ms. Trace:
I20260709 12:26:22.284265 31144 rpcz_store.cc:276] 0709 12:26:20.492487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.127614 (+1635127us) service_pool.cc:224] Handling call
0709 12:26:22.127936 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.262783 (+134847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.262995 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.285663 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:22.239792 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1207) took 1745 ms. Trace:
I20260709 12:26:22.286602 31139 rpcz_store.cc:276] 0709 12:26:20.494002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.237086 (+1743084us) service_pool.cc:224] Handling call
0709 12:26:22.237509 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.239369 (+  1860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.239407 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.239465 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:22.239509 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.239579 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.239724 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.288154 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:22.289590 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:22.263401 31693 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:22.195910 31697 rpcz_store.cc:276] 0709 12:26:20.826556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.826679 (+   123us) service_pool.cc:224] Handling call
0709 12:26:22.125698 (+1299019us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:22.291682 30958 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.033s	sys 0.021s
I20260709 12:26:22.291754 31068 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:22.292265 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 1.38149e+06us
I20260709 12:26:22.134640  5017 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:22.264907  4818 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.579s	user 0.010s	sys 0.002s
I20260709 12:26:22.137297  5009 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.355335 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:22.180505  4673 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.359107 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2915) took 2143 ms. Trace:
I20260709 12:26:22.364667 30984 rpcz_store.cc:276] 0709 12:26:20.215304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.291166 (+2075862us) service_pool.cc:224] Handling call
0709 12:26:22.358134 (+ 66968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.358744 (+   610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.358773 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.358815 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:22.358871 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.358930 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.359046 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:22.336303 30957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.010s	sys 0.030s
W20260709 12:26:22.370990 30957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.010s	sys 0.030s
W20260709 12:26:22.294974  4863 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.451s	user 0.013s	sys 0.001s
W20260709 12:26:22.383608 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.003s	sys 0.040s
W20260709 12:26:22.384027 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.003s	sys 0.040s
W20260709 12:26:22.376570 31106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.012s	sys 0.032s
W20260709 12:26:22.386729 31106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.012s	sys 0.032s
W20260709 12:26:22.358958  4957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.049s	sys 0.082s
W20260709 12:26:22.377084 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.007s	sys 0.036s
W20260709 12:26:22.388315 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.007s	sys 0.036s
W20260709 12:26:22.388870 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.389559 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.381939 30958 connection.cc:256] RPC call timeout handler was delayed by -1.1840436219999901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
W20260709 12:26:22.395011 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.286706  4896 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.406924 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:22.412567 30984 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
W20260709 12:26:22.413530  5021 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.361s	user 0.000s	sys 0.000s
I20260709 12:26:22.413707 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:22.414156  4972 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:22.414619 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3050) took 1723 ms. Trace:
I20260709 12:26:22.415122 31058 rpcz_store.cc:276] 0709 12:26:20.691715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.691840 (+   125us) service_pool.cc:224] Handling call
0709 12:26:20.698512 (+  6672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.698852 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.698887 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.707090 (+  8203us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.908071 (+200981us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:20.908113 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.908173 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.414410 (+1506237us) raft_consensus.cc:1685] finished
0709 12:26:22.414430 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.414566 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9334,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":55295,"thread_start_us":200671,"threads_started":1,"wal-append.queue_time_us":1231163}
I20260709 12:26:22.416942  5009 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.286208  4952 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.418354 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:22.418946  4952 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:22.388069  4957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.049s	sys 0.082s
I20260709 12:26:22.420751 31059 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:22.420874 31067 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:22.421936 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.414868 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:22.423358 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:22.424110 31063 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:22.424345 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:22.408660 30989 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.010s	sys 0.003s
I20260709 12:26:22.377724  4563 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:22.426337  4563 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.409801 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:22.426712  4563 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.427397 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:22.428952 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:22.429814  5036 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:22.430195  5036 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:22.430624  5036 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:22.430809  4785 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:22.425633  5009 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.431700  5009 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:22.432229  4993 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.966s	user 0.005s	sys 0.015s
W20260709 12:26:22.408391 31694 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.000s	sys 0.012s
W20260709 12:26:22.435081 31694 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.000s	sys 0.012s
W20260709 12:26:22.435217 30989 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.010s	sys 0.003s
I20260709 12:26:22.380622  4853 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.435956 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2911) took 2329 ms. Trace:
I20260709 12:26:22.440434 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2762) took 2305 ms. Trace:
I20260709 12:26:22.472760 31062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
W20260709 12:26:22.438005 31662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.005s	sys 0.009s
W20260709 12:26:22.586637 31662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.005s	sys 0.009s
I20260709 12:26:22.587899 31062 raft_consensus.cc:606] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:22.588641 31062 raft_consensus.cc:614] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:22.440672 30989 rpcz_store.cc:276] 0709 12:26:20.106117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.689222 (+583105us) service_pool.cc:224] Handling call
0709 12:26:20.689402 (+   180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.689531 (+   129us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:20.899083 (+209552us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.435663 (+1536580us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.435710 (+    47us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:22.435749 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.435807 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.435909 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":805785,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":4089,"raft.queue_time_us":211756,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":12801,"thread_start_us":1720942,"threads_started":3,"wal-append.queue_time_us":1133093}
I20260709 12:26:22.567035 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:22.443912 31065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.020s	sys 0.036s
I20260709 12:26:22.475888 31692 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:22.293058  4949 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.385795  4815 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:26:22.484997 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2913) took 2275 ms. Trace:
W20260709 12:26:22.592272 31065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.020s	sys 0.054s
W20260709 12:26:22.523443  4996 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 2.114s	user 0.010s	sys 0.020s
W20260709 12:26:22.458878 31109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.009s	sys 0.009s
I20260709 12:26:22.382728  4818 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:22.539083 30986 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 1200) took 3414 ms (client timeout 2588 ms). Trace:
W20260709 12:26:22.601127 30986 rpcz_store.cc:269] 0709 12:26:19.125103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.149463 (+ 24360us) service_pool.cc:224] Handling call
0709 12:26:22.538761 (+3389298us) spinlock_profiling.cc:243] Waited 3.38 s on lock 0x7b5400226854. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:22.538966 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7765126528}
I20260709 12:26:22.487592 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:22.540021 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1323) took 1839 ms. Trace:
I20260709 12:26:22.602808 31693 rpcz_store.cc:276] 0709 12:26:20.700855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.700986 (+   131us) service_pool.cc:224] Handling call
0709 12:26:20.770343 (+ 69357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.290819 (+1520476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.539783 (+248964us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540011b7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:22.539951 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3377070336}
I20260709 12:26:22.513294  5009 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.461300  5032 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.543813  4978 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:22.441597 31063 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:22.489569 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3047) took 1798 ms. Trace:
I20260709 12:26:22.545320 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2891) took 4197 ms. Trace:
I20260709 12:26:22.490602 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:22.552893  4994 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 2.211s	user 0.024s	sys 0.009s
I20260709 12:26:22.546741  5035 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:22.442103 30988 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.014s	sys 0.001s
W20260709 12:26:22.618476 30988 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.015s	sys 0.007s
I20260709 12:26:22.405089  4828 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:22.555230  5026 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.425s	user 0.001s	sys 0.000s
W20260709 12:26:22.541636 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 272674us
I20260709 12:26:22.387600  4863 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.626569 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:22.627486  4863 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.627767 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:22.536752  4915 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.628463  4780 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:22.628957  4915 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.580636 31142 rpcz_store.cc:276] 0709 12:26:20.134893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.903794 (+768901us) service_pool.cc:224] Handling call
0709 12:26:20.904107 (+   313us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:20.936044 (+ 31937us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:20.936076 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:21.871426 (+935350us) log.cc:844] Serialized 52 byte log entry
0709 12:26:22.409112 (+537686us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:22.409157 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.409240 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.439264 (+ 30024us) raft_consensus.cc:1685] finished
0709 12:26:22.440225 (+   961us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.440373 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31514,"raft.queue_time_us":34886,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6083,"thread_start_us":435488,"threads_started":1,"wal-append.queue_time_us":517653}
I20260709 12:26:22.539184 30987 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:22.554677 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:22.519568 31691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1324) took 2412 ms (client timeout 1548 ms). Trace:
I20260709 12:26:22.511726 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:22.482074 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:22.558966 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:22.582453  4931 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.025s	sys 0.024s
I20260709 12:26:22.354231  5017 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.608861  4918 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.630152 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.592108 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:22.643481  5000 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 2.188s	user 0.012s	sys 0.035s
I20260709 12:26:22.643663 31063 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
W20260709 12:26:22.644107 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 52099.2us
I20260709 12:26:22.622725  5035 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.645455 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2590) took 2789 ms. Trace:
I20260709 12:26:22.645696 31137 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
W20260709 12:26:22.598605 31109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.010s	sys 0.013s
I20260709 12:26:22.646894 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:22.599221 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3051) took 1903 ms. Trace:
W20260709 12:26:22.462651  5023 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260709 12:26:22.583268  5016 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.468s	user 0.000s	sys 0.001s
W20260709 12:26:22.605952 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.607959 30986 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:26:22.654974 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2667) took 2176 ms. Trace:
I20260709 12:26:22.513715  5033 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:22.613301  4862 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.613590  4973 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:22.614785 31068 rpcz_store.cc:276] 0709 12:26:20.690688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.690889 (+   201us) service_pool.cc:224] Handling call
0709 12:26:20.693563 (+  2674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.338800 (+1645237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.489163 (+150363us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54002cd8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:22.489463 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3674947328}
I20260709 12:26:22.615849 30992 rpcz_store.cc:276] 0709 12:26:18.347933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.350533 (+  2600us) service_pool.cc:224] Handling call
0709 12:26:18.505690 (+155157us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.664207 (+158517us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:18.664254 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.672189 (+  7935us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.922093 (+249904us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:18.923337 (+  1244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.923408 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.031144 (+2107736us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:22.234309 (+1203165us) raft_consensus.cc:1685] finished
0709 12:26:22.234330 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.545087 (+310757us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400023ad0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:22.545229 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":217172,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":431247872,"thread_start_us":323423,"threads_started":2,"wal-append.queue_time_us":308576}
I20260709 12:26:22.657629 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:22.596843  4980 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.594841  4942 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.597954  5031 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:22.621810 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.458889  4853 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.528426 31141 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:22.643119  4896 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.644806 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:22.669922 31698 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 1314) took 2848 ms (client timeout 1509 ms). Trace:
W20260709 12:26:22.670454 31698 rpcz_store.cc:269] 0709 12:26:19.821434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.821565 (+   131us) service_pool.cc:224] Handling call
0709 12:26:22.669710 (+2848145us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b5400511b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:22.669844 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771532544}
I20260709 12:26:22.670990 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:22.516599  5012 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.455s	user 0.000s	sys 0.001s
W20260709 12:26:22.648202 31691 rpcz_store.cc:269] 0709 12:26:20.106896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.109524 (+  2628us) service_pool.cc:224] Handling call
0709 12:26:22.519339 (+2409815us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54006a09d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:22.519488 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751448320}
I20260709 12:26:22.677263 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1642) took 2266 ms. Trace:
I20260709 12:26:22.677619 31060 rpcz_store.cc:276] 0709 12:26:20.410679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.411249 (+   570us) service_pool.cc:224] Handling call
0709 12:26:20.437244 (+ 25995us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.458830 (+ 21586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.458876 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.469513 (+ 10637us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.695200 (+225687us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:20.695240 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.695302 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.677039 (+1981737us) raft_consensus.cc:1685] finished
0709 12:26:22.677061 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.677189 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3660,"raft.queue_time_us":4112,"spinlock_wait_cycles":21502720,"thread_start_us":190479,"threads_started":1,"wal-append.queue_time_us":1448565}
I20260709 12:26:22.602056  4877 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.647600 31065 rpcz_store.cc:276] 0709 12:26:20.695461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.696491 (+  1030us) service_pool.cc:224] Handling call
0709 12:26:20.696861 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.697118 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:20.908275 (+211157us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.560966 (+1652691us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b18003d37e0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87752a6 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1
0709 12:26:22.562795 (+  1829us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b18003d37e0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:22.598291 (+ 35496us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.598357 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:22.598408 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.599055 (+   647us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.599165 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20084,"prepare.queue_time_us":1355417,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":69954,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":44985,"spinlock_wait_cycles":49002624,"thread_start_us":1842326,"threads_started":3,"wal-append.queue_time_us":1064736}
I20260709 12:26:22.591656 30991 rpcz_store.cc:276] 0709 12:26:20.209679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.757704 (+548025us) service_pool.cc:224] Handling call
0709 12:26:20.833885 (+ 76181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.838971 (+  5086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.839003 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.839513 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.050839 (+1211326us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:22.050885 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.050974 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.484766 (+433792us) raft_consensus.cc:1685] finished
0709 12:26:22.484799 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.484935 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2128,"raft.queue_time_us":37807,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":21347,"thread_start_us":1211043,"threads_started":1,"wal-append.queue_time_us":1474440}
I20260709 12:26:22.651486  4852 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:22.631173 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1198) took 2962 ms. Trace:
I20260709 12:26:22.680542  4853 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.653784  4754 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:22.681274  4754 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.681011 31136 rpcz_store.cc:276] 0709 12:26:19.668750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.711039 (+ 42289us) service_pool.cc:224] Handling call
0709 12:26:19.711355 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.711712 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.711743 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.712204 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.198081 (+485877us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:20.198162 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.198328 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.552312 (+2353984us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:22.630744 (+ 78432us) raft_consensus.cc:1685] finished
0709 12:26:22.630776 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.631073 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2841,"raft.queue_time_us":1650,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9390,"spinlock_wait_cycles":7306112,"thread_start_us":425637,"threads_started":1,"wal-append.queue_time_us":629378}
I20260709 12:26:22.655274 30986 rpcz_store.cc:276] 0709 12:26:20.478897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.604434 (+2125537us) service_pool.cc:224] Handling call
0709 12:26:22.604842 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.654701 (+ 49859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.654919 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.683523  4754 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:22.651537  5034 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.633963  4992 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:22.657696 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:22.650115  4931 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.025s	sys 0.025s
I20260709 12:26:22.614261  5042 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:22.720925  5042 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.666005 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2766) took 2387 ms. Trace:
I20260709 12:26:22.647650  5047 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:22.631112  4818 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.662776  4908 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:26:22.684438 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2670) took 2197 ms. Trace:
I20260709 12:26:22.691418  4980 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:22.686640 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:22.645843 31143 rpcz_store.cc:276] 0709 12:26:19.855654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.858566 (+  2912us) service_pool.cc:224] Handling call
0709 12:26:19.858907 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.859305 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.859345 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.874678 (+ 15333us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.190640 (+315962us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:20.222177 (+ 31537us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.240602 (+ 18425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.240702 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.642868 (+2402166us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:22.645193 (+  2325us) raft_consensus.cc:1685] finished
0709 12:26:22.645283 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.645402 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32487,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":1963,"raft.queue_time_us":20260,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":12799,"spinlock_wait_cycles":65969536,"thread_start_us":238856,"threads_started":1,"wal-append.queue_time_us":480693}
I20260709 12:26:22.680786 30991 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:26:22.735801 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2669) took 2249 ms. Trace:
I20260709 12:26:22.641608  5004 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.736145 30991 rpcz_store.cc:276] 0709 12:26:20.486756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.680250 (+2193494us) service_pool.cc:224] Handling call
0709 12:26:22.680581 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.735460 (+ 54879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.735746 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:22.666738  5031 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:22.617681  5032 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.663470  4876 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:26:22.707137  4915 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:22.711318 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:22.714812 30958 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:22.716144 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 58538.6us
I20260709 12:26:22.656872  4957 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:22.656625  5030 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:22.722084 31141 rpcz_store.cc:276] 0709 12:26:20.278969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.045324 (+766355us) service_pool.cc:224] Handling call
0709 12:26:22.528017 (+1482693us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b5800004598. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876feee 000000000036eb39 00007f2ef07e31e0 00007f2ef08ac0c4 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:26:22.528215 (+   198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.665751 (+137536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.665953 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16627712}
I20260709 12:26:22.756754 31141 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:22.657097  4771 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:22.659368  4973 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:22.763585 31135 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:22.765623 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2754) took 2642 ms. Trace:
I20260709 12:26:22.766059 31138 rpcz_store.cc:276] 0709 12:26:20.123392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.123906 (+   514us) service_pool.cc:224] Handling call
0709 12:26:20.193097 (+ 69191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.651892 (+458795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.765351 (+2113459us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540024ac54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:22.765559 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":993118592}
I20260709 12:26:22.725920  5047 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:22.730994 30990 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2650) took 3805 ms (client timeout 1710 ms). Trace:
I20260709 12:26:22.647557 31137 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:22.732122 30989 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:26:22.737716 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 51207.7us
I20260709 12:26:22.704334  4937 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:22.739535 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 310631us
I20260709 12:26:22.774582 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:22.770625 30990 rpcz_store.cc:269] 0709 12:26:18.925260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.948889 (+ 23629us) service_pool.cc:224] Handling call
0709 12:26:22.730760 (+3781871us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b540014e6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:22.730910 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1819875072}
I20260709 12:26:22.776458 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:22.738847  5031 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:22.778630 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:22.722231  4563 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:22.747316 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.728782  4673 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:22.752027 31142 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:22.730021 30986 rpcz_store.cc:276] 0709 12:26:20.487041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.683553 (+2196512us) service_pool.cc:224] Handling call
0709 12:26:22.683829 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.684117 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.684150 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.684187 (+    37us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:22.684226 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.684286 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.684388 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":342,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":21064}
I20260709 12:26:22.812021 30986 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:22.734678  4980 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:22.826747 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48203.4us
I20260709 12:26:22.813910 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:22.739102  4877 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:22.663204  5033 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.836431 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:22.830807  4980 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:22.838543  4963 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.839229  4963 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.810832  5049 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:22.843237  5049 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:22.843367  4852 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:22.843665  5049 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:22.844017  4852 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.844539  4852 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:22.846323  5033 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.847600 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1326) took 2076 ms. Trace:
I20260709 12:26:22.848027 31694 rpcz_store.cc:276] 0709 12:26:20.770792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.770931 (+   139us) service_pool.cc:224] Handling call
0709 12:26:20.839801 (+ 68870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.951552 (+111751us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:21.875797 (+924245us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.458839 (+583042us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.458895 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:22.458949 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.847368 (+388419us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540076ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:22.847415 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.847518 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1090407,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":58073,"raft.queue_time_us":25272,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":13366,"spinlock_wait_cycles":256456448,"thread_start_us":1462975,"threads_started":2,"wal-append.queue_time_us":782592}
I20260709 12:26:22.852587  4949 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.853084  4991 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.854090  4933 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.725284  4853 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:22.854737  5017 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.854964  4853 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:22.795058 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.773674  4815 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:22.858860  4815 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.859304  5032 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:22.803870  4985 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:22.771323  5030 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.861016  4985 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:22.861472  4985 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:22.806860 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.863270 31065 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:22.863023  4973 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.718564  5034 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:22.864058  4902 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.000s
W20260709 12:26:22.809482  5019 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:22.864473 31144 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:22.865598  5034 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.808307 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:22.827107 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2672) took 2034 ms. Trace:
I20260709 12:26:22.771615  4957 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.866353 30986 rpcz_store.cc:276] 0709 12:26:20.792426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.811413 (+2018987us) service_pool.cc:224] Handling call
0709 12:26:22.811776 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.826863 (+ 15087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.827052 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:22.867748 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59540.6us
I20260709 12:26:22.868029  4957 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:22.809975  4942 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:26:22.869052 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:22.869232  4942 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:22.869648  4942 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:22.871189 31144 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:22.873471 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:22.870148 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:22.773876  4754 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:22.859885  5032 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:22.862488  5030 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:22.835134 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.885372 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.886129 31062 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:22.887161 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.873152 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2600) took 2368 ms. Trace:
I20260709 12:26:22.876931 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:22.878422 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.873642 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2602) took 2301 ms. Trace:
I20260709 12:26:22.870887  4937 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.889575 31139 rpcz_store.cc:276] 0709 12:26:20.504987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.672264 (+2167277us) service_pool.cc:224] Handling call
0709 12:26:22.672620 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.872333 (+199713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.872361 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.872398 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:22.872433 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.872940 (+   507us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540006e0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:22.872979 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.873097 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458865536}
I20260709 12:26:22.815021  5035 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:22.774235  5009 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.100s
I20260709 12:26:22.890244 31136 rpcz_store.cc:276] 0709 12:26:20.571946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.682809 (+2110863us) service_pool.cc:224] Handling call
0709 12:26:22.683086 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.872847 (+189761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.872877 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.872915 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:22.872951 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.873433 (+   482us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54003f48d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:22.873474 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.873592 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435978496}
I20260709 12:26:22.892874 31059 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:22.902591 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.893537 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:22.831207  4969 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:22.896872 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 271005us
I20260709 12:26:22.903417 31061 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:22.872000  4962 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:22.859344  4815 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.872164  4896 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:22.898391 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.876173  4931 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:22.881796  4887 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:22.864073  4973 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:22.907747 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:22.914134 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:22.919988 31067 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:22.896559  4933 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:22.921079 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:22.864481  4902 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.911850  4969 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:22.856135  4853 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.911417  4962 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:22.870476  4942 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.918597  4931 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:22.931440  4910 consensus_meta.cc:287] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.000s
I20260709 12:26:22.929803  5017 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:22.939914  5042 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.940352  4942 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:22.929958  4991 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:22.949033  4877 consensus_queue.cc:1175] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" responder_term: 15 status { last_received { term: 14 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:22.936821  4962 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.936693  4969 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.903954  4985 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.947850  4931 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:22.909886  5047 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.959853  5017 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:22.995100  5017 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:22.971168  5040 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.999606  5040 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.002787  5047 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:22.988409  5038 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:23.006325  5038 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:23.006594  4985 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:23.007511  5038 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.009166  5038 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:22.961635  4991 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:23.014721  4991 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:23.016981  5040 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:22.988895  4980 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:22.989624  4963 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:22.993577  4952 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.000s
I20260709 12:26:22.985737  5004 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.023885  4980 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:22.998782  5036 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.568s	user 0.013s	sys 0.000s
I20260709 12:26:23.026899  5036 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.029573  5036 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:23.051347  4969 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.056206  4969 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.059051  5031 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.060830  5049 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.061714  5031 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.064245  5049 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:23.108409 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 260793us
I20260709 12:26:23.111160 30983 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:23.111438 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 217850us
I20260709 12:26:23.113193 30986 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [17.11-19.12]
W20260709 12:26:23.115293 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.118990 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:23.120692 31109 connection.cc:256] RPC call timeout handler was delayed by -1.8127664469998308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 12:26:23.112999 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.126279 31697 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:23.126784 31067 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:23.127769 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.128379 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.128634  5038 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:23.129016  4563 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.129457 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.121224 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.136770 31063 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:22.953930  4910 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.146025  4921 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:23.153060 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 380167us
I20260709 12:26:23.153829 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.154377  4969 raft_consensus.cc:2764] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:23.178869 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.126904  4991 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:23.156155 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.157696 31068 raft_consensus.cc:2410] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:26:23.158238  5031 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.162407 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.164885 31060 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:23.171166 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:23.174664 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:23.171186 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.176807 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2909) took 3071 ms. Trace:
I20260709 12:26:23.177110  4902 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.144080  4980 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.180568 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2673) took 2385 ms. Trace:
I20260709 12:26:23.181627 30992 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:26:23.181762  4910 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:23.183640 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 70952.7us
I20260709 12:26:23.129648  5038 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:23.185233 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.187347  5004 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:23.146014 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.191349 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.210811 31065 raft_consensus.cc:2393] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:23.212162  4962 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.214746  4980 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:23.215857 31664 connection.cc:256] RPC call timeout handler was delayed by -1.4506550299997798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 12:26:23.216673 31664 connection.cc:256] RPC call timeout handler was delayed by -1.3254712489997473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25880000000000003s for a response
I20260709 12:26:23.199273 30986 rpcz_store.cc:276] 0709 12:26:20.794830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.867174 (+2072344us) service_pool.cc:224] Handling call
0709 12:26:22.867518 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:23.113794 (+246276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.180123 (+ 66329us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400227254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:23.180445 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563151744}
I20260709 12:26:23.218588 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.198659 30985 rpcz_store.cc:276] 0709 12:26:20.105676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.464269 (+358593us) service_pool.cc:224] Handling call
0709 12:26:23.176375 (+2712106us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54000244d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:23.176591 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3892506880}
I20260709 12:26:23.220213 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.220350 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.209211 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.225709  5040 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:23.226544  5040 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.236058 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.237664 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.229863 31061 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
I20260709 12:26:23.238899  4771 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.239154 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.239696  4771 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.240732 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.241369 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.241676  4931 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.242403  4921 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.243345  4771 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:26:23.245065  4921 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:23.241906 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.265190 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:23.264094 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.264485 31063 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 10.
I20260709 12:26:23.245227  4931 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:23.278294 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.259995 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2664) took 2781 ms. Trace:
W20260709 12:26:23.258493 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.218152  4980 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:23.282927 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.260787 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1204) took 2767 ms. Trace:
I20260709 12:26:23.253250 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.283957 31061 raft_consensus.cc:2410] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:23.284168 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.260799  4991 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:23.258754 31142 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:23.255435 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 967) took 2351 ms. Trace:
I20260709 12:26:23.280303 30989 rpcz_store.cc:276] 0709 12:26:20.478152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.599436 (+2121284us) service_pool.cc:224] Handling call
0709 12:26:22.731704 (+132268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.771900 (+ 40196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.259601 (+487701us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540014e6d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:23.259774 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302103808}
I20260709 12:26:23.269520 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:23.303978 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2916) took 2958 ms. Trace:
I20260709 12:26:23.305567  5004 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:23.250190 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:23.317112 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.306689  4892 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:23.319749 31139 raft_consensus.cc:2410] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:23.319839  4892 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:23.320426  4892 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:23.321220 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:23.307786 31061 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:23.322009 31139 raft_consensus.cc:2376] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19. Re-sending same reply.
I20260709 12:26:23.322530 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.323207 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.323684 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.323932 31061 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:23.307570  4969 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:23.324612  4918 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:23.325235  4969 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:23.325616  4969 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:23.325759 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:23.326673 31693 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:23.327786 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:23.327936 31136 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:23.309927 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.329927 31063 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
W20260709 12:26:23.325762 31690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51362 (request call id 1320) took 2660 ms (client timeout 1568 ms). Trace:
I20260709 12:26:23.331127 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2661) took 2853 ms. Trace:
W20260709 12:26:23.331024 31690 rpcz_store.cc:269] 0709 12:26:20.664873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.669466 (+  4593us) service_pool.cc:224] Handling call
0709 12:26:23.325427 (+2655961us) spinlock_profiling.cc:243] Waited 2.44 s on lock 0x7b54004443d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:23.325574 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5612333440}
I20260709 12:26:23.331569 30983 rpcz_store.cc:276] 0709 12:26:20.477411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.550110 (+2072699us) service_pool.cc:224] Handling call
0709 12:26:22.550431 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.112071 (+561640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.330883 (+218812us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b5400312c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:23.331071 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287806848}
I20260709 12:26:23.332304 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.332751 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.332921 31059 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:23.333673 31033 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:23.335258 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.335633 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.336021 31067 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:23.337338 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.337476 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2656) took 4410 ms. Trace:
I20260709 12:26:23.244920 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.268325 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2912) took 3058 ms. Trace:
I20260709 12:26:23.340322  5036 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:23.340921  5036 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:23.318384 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2668) took 2839 ms. Trace:
I20260709 12:26:23.342334 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:23.279774 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:23.328006 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.263589 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.343703 31696 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.306778 31137 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:23.337983 30987 rpcz_store.cc:276] 0709 12:26:18.926972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.376595 (+449623us) service_pool.cc:224] Handling call
0709 12:26:22.538933 (+3162338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.647913 (+108980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.337076 (+689163us) spinlock_profiling.cc:243] Waited 3.16 s on lock 0x7b5400226850. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:23.337289 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7272073984}
I20260709 12:26:23.286623 31140 rpcz_store.cc:276] 0709 12:26:20.493256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.128491 (+1635235us) service_pool.cc:224] Handling call
0709 12:26:22.130067 (+  1576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.134453 (+  4386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.134496 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.134536 (+    40us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:22.134580 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.260314 (+1125734us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b540029efd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:23.260372 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.260585 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9638272}
I20260709 12:26:23.318079 31058 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:23.347386 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.327797 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:23.348223 30987 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:23.349629 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:23.342368 30992 rpcz_store.cc:276] 0709 12:26:20.479359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.660055 (+2180696us) service_pool.cc:224] Handling call
0709 12:26:23.181014 (+520959us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.211405 (+ 30391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.318152 (+106747us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b54000244d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:23.318324 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189988736}
I20260709 12:26:23.350718 30989 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.351298 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:23.352216 30992 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:23.318961 31698 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:23.250941  5031 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.250769 31106 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:23.359387 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 114679us
W20260709 12:26:23.359483 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 108941us
I20260709 12:26:23.360080  4908 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:23.360956  5017 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:23.361078 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:23.358294  4918 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:23.362341  4918 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:23.362576  5061 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:23.360417 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:23.363711  5036 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.317955  5038 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.370929  4949 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:23.312837 30984 rpcz_store.cc:276] 0709 12:26:20.344957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.433980 (+2089023us) service_pool.cc:224] Handling call
0709 12:26:22.434334 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.434594 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:22.737681 (+303087us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.302757 (+565076us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.303164 (+   407us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.303346 (+   182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.303400 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:23.303497 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.303610 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.303784 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":497457,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":1944,"raft.queue_time_us":38988,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":270,"thread_start_us":773357,"threads_started":2,"wal-append.queue_time_us":560549}
I20260709 12:26:23.353566  4969 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.374765  4921 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:23.375296  4969 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:23.378847 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:23.380893  4921 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:23.381383  4921 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.321832  4892 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.382869  4892 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:23.382992 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.383636 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.384881 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.385524 31137 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:23.381788  4921 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:23.386649 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.387712 31144 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:23.387871  5004 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:23.388770  5004 consensus_queue.cc:1243] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:23.389297  5004 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:23.389979  5004 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:23.390338 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.391436 31143 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 22: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:23.392938 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.394155 31137 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 22: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:23.313647 31689 rpcz_store.cc:276] 0709 12:26:20.904144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.904613 (+   469us) service_pool.cc:224] Handling call
0709 12:26:20.904981 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.263494 (+1358513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.254781 (+991287us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540055c654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:23.255181 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821394688}
I20260709 12:26:23.395835 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.338841 30988 rpcz_store.cc:276] 0709 12:26:20.209395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.714664 (+505269us) service_pool.cc:224] Handling call
0709 12:26:21.880540 (+1165876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.880810 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:21.892606 (+ 11796us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.264175 (+1371569us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b1800393b80. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a
0709 12:26:23.267637 (+  3462us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.267719 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:23.267797 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.267943 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.268122 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12277,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":2856,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":224,"spinlock_wait_cycles":6148864,"thread_start_us":721683,"threads_started":1,"wal-append.queue_time_us":722631}
I20260709 12:26:23.390606  5004 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:23.398727 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.399441  5004 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.401141  5004 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:23.402310  4918 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.406816  4918 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.306062  4962 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.399338 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.437172 31142 raft_consensus.cc:2410] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:26:23.436198 31696 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:23.451017 30988 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:23.452112 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:23.390365  5040 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:23.455277 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.450037  5048 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:23.467044  5048 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:23.468003 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 89298.1us
I20260709 12:26:23.469019 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.474726  4790 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:23.467453 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 84634us
I20260709 12:26:23.425416  5031 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:23.478148 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:23.452457 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2674) took 2621 ms. Trace:
I20260709 12:26:23.479517 30986 rpcz_store.cc:276] 0709 12:26:20.825985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.401443 (+2575458us) service_pool.cc:224] Handling call
0709 12:26:23.402039 (+   596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.440468 (+ 38429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.440510 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.447173 (+  6663us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:23.447234 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.447313 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.447443 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.482456 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:23.490098 31140 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.492420 30987 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:23.509606 30990 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 14. Current term is 15. Ops: [12.11-14.12]
W20260709 12:26:23.482782 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 121851us
I20260709 12:26:23.518394 30988 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:23.520455 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:23.523546 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 256673us
W20260709 12:26:23.528155 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59380.1us
I20260709 12:26:23.529093 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.529670 31058 raft_consensus.cc:2376] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18. Re-sending same reply.
I20260709 12:26:23.540848 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2675) took 2709 ms. Trace:
I20260709 12:26:23.530640  5048 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:23.554988 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.551678 30992 rpcz_store.cc:276] 0709 12:26:20.826339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.407977 (+2581638us) service_pool.cc:224] Handling call
0709 12:26:23.408378 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.408758 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:23.409456 (+   698us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.533970 (+124514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.534062 (+    92us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:23.534277 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.534683 (+   406us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.535239 (+   556us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":218,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":9306,"thread_start_us":116404,"threads_started":1}
I20260709 12:26:23.552977  5036 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:23.562266 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.571595 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:23.571918 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.572891 31067 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:23.568740  5030 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:23.574899 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:23.572891  5040 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:26:23.577381 30876 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2439e7934b98446ca4e7d82381e8a499 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e9cccf7eb2a545ef8c2169b67ff261bf is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:23.583863 31064 raft_consensus.cc:2410] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 12:26:23.594177 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 102588us
I20260709 12:26:23.596933 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.598285 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:23.605429 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.608006 30958 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:23.617650 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 95261.4us
I20260709 12:26:23.626564  5048 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:23.601302  5040 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:26:23.627357  5048 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:23.628268  4779 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:23.628726  4779 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:23.629061  4779 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.628984  5048 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.627394 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.630518 30988 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:23.632278 31064 raft_consensus.cc:2410] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:26:23.633306 31661 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:23.635639 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.638062  4949 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:23.644447 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 95632.1us
I20260709 12:26:23.622720  4892 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:23.650476 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:23.630843  4815 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.005s
I20260709 12:26:23.627643  5040 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:26:23.655359  5062 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:26:23.655848 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1212) took 3073 ms. Trace:
I20260709 12:26:23.642913  4790 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:23.656338 30990 rpcz_store.cc:276] 0709 12:26:20.582561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.776389 (+2193828us) service_pool.cc:224] Handling call
0709 12:26:22.808662 (+ 32273us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:23.510103 (+701441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.650125 (+140022us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b540011fdd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:23.655790 (+  5665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1556304256}
I20260709 12:26:23.658167 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.654376  5040 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:23.658409  4790 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.680163 31062 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:26:23.693018 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 105218us
I20260709 12:26:23.697283 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.685926  4790 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.699594 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.659583  4815 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.705874 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.707477 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.708997 31058 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:23.706478  5030 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.711150  5030 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.712474 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.713349 31068 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:23.713591  5030 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.714187  5035 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:23.714428 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.714284 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.715083 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:23.715682 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.720682 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.721216 31063 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.723243 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.717536  5035 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.724135  5035 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:23.724831 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.725804 31061 raft_consensus.cc:2410] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:26:23.724236  5069 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:26:23.727878  5033 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:23.726936 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:23.724418  5065 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 12:26:23.731266 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.731109  4921 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.731600  4673 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:23.732501 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.732013 31067 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
W20260709 12:26:23.736083 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58450.8us
I20260709 12:26:23.736455  4921 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:23.737273 31061 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:23.737479 31106 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:23.738299 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.738878 31061 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.741525  4876 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:23.742769  4673 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:23.743229  4673 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.743539  4876 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:23.744442  4876 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.745975  4815 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:23.729219  4852 consensus_meta.cc:287] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.830s	user 0.010s	sys 0.012s
I20260709 12:26:23.734584 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.746785  4852 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.749028  4852 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.749630 31064 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:23.749886 30985 raft_consensus.cc:2368] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:26:23.743635  4673 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.765300 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:23.766676  5040 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.768054  5040 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.770545 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.771781 31061 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
W20260709 12:26:23.772218 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.772657 30989 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
W20260709 12:26:23.772367 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45425.8us
I20260709 12:26:23.774606 30983 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:23.728273  5033 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:23.776547 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:23.776623  4942 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:23.709851  4949 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:26:23.777197  4918 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:26:23.783193  4949 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:23.783569  4949 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:23.778999 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44511.1us
I20260709 12:26:23.784582 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1208) took 3288 ms. Trace:
I20260709 12:26:23.789407 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:23.790984 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:23.791610 31060 raft_consensus.cc:2376] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17. Re-sending same reply.
I20260709 12:26:23.778898  5033 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:23.790865 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:23.794037 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 85506.3us
I20260709 12:26:23.797283  4780 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:26:23.787313 30991 rpcz_store.cc:276] 0709 12:26:20.496132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.736873 (+2240741us) service_pool.cc:224] Handling call
0709 12:26:22.760697 (+ 23824us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:22.761151 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.761196 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:22.761919 (+   723us) log.cc:844] Serialized 52 byte log entry
0709 12:26:23.269759 (+507840us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:23.269874 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.269964 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.780440 (+510476us) raft_consensus.cc:1685] finished
0709 12:26:23.780460 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.784412 (+  3952us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1009,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":11596,"thread_start_us":353410,"threads_started":1,"wal-append.queue_time_us":639042}
W20260709 12:26:23.800635 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.797966  5076 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:26:23.809556 31140 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:23.799181  4852 raft_consensus.cc:2764] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:26:23.812614 31062 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:23.817615 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52500.2us
I20260709 12:26:23.820005  4942 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:23.784571  4949 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.826748 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:23.829015  5075 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.164s	user 0.004s	sys 0.000s
I20260709 12:26:23.822580 30958 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:23.835024  5060 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:26:23.835336 31063 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:26:23.833820  4969 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:26:23.838320  4969 consensus_queue.cc:1243] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:23.823480  4942 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.833465  4876 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:23.866532 31140 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:23.838800  4892 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:26:23.868961 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.838723  4918 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:23.868723  4892 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:23.867254  5031 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:23.882629 31062 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:23.847555  4942 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:23.887264 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58524.7us
W20260709 12:26:23.886713 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57906.3us
I20260709 12:26:23.871652  4918 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:23.835955  5017 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:26:23.894920 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:23.871625  4969 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.867086  5060 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:26:23.895507 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:23.897526 31696 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:23.891923  5082 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:26:23.899255  4918 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:23.839397  5030 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:26:23.914309 31135 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:23.914601 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.918354  4969 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:23.922338 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 97364.3us
I20260709 12:26:23.923506 31694 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:23.926517  4892 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.927748  4991 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:23.937057 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:23.940719 31138 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:23.938472  4949 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:23.929790  4790 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:23.945062 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 79431.8us
I20260709 12:26:23.955016 31144 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:23.956432 31691 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:23.909065  5031 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:23.971546 31106 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:23.971675 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:23.973114 31691 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:23.948768  5035 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:23.945727  4991 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:23.975384 31698 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:23.978359 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:23.978837  5035 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:23.980046  5035 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:23.981462  4991 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:23.982968  4991 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:23.984000 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:23.986716  5035 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:23.992841 31696 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:23.997587  5060 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:24.003674 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 115652us
I20260709 12:26:24.005616 31140 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:24.004973 31661 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:24.009235 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:24.017833 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:24.019906 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.021870 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.022636 31696 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:24.019476 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.023105 31694 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.022503 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:23.988118  5035 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.026715 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.027922 31142 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:24.027210 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59577.1us
I20260709 12:26:24.031066 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.035179 31692 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:26:24.035535 31692 pending_rounds.cc:85] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 10
I20260709 12:26:24.035735 31692 pending_rounds.cc:107] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:26:24.036003 31692 raft_consensus.cc:2889] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.036314 31692 pending_rounds.cc:107] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:26:24.036533 31692 raft_consensus.cc:2889] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:24.038592 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 49988.6us
I20260709 12:26:24.039363 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.039978 31137 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:24.040925 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.041096 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:24.041460 31136 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:24.041778  5017 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:24.042395 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:24.043205 30983 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:24.043876  5004 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:24.044059 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:23.999680  4779 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.044548  5004 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.045518 30989 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:24.046186  4779 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:24.048341 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.049230  4852 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:24.049470  5004 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:24.050426  4952 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:24.051497  4962 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:24.051862  4962 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:24.052806  4962 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.050437 31689 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:24.057214  5040 raft_consensus.cc:2764] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:24.056655 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.060817  5031 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:24.061324  5031 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:24.062316  5031 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.032819 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.065111  5004 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:24.033564 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:24.027947  4892 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.092769 31142 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:24.093828 31140 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:26:24.091655 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.015129  4991 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:26:24.104543 31143 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:24.100625 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.106803 31698 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:24.115366 31109 leader_election.cc:400] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:26:24.115688 31109 leader_election.cc:403] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:24.125116 31690 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:24.101120  5017 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:24.124058 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 105753us
I20260709 12:26:24.134337 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:24.098888  5004 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
W20260709 12:26:24.137578 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 98311.6us
W20260709 12:26:24.137722 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 167277us
I20260709 12:26:24.115592  5035 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:24.137549  4969 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:24.144279 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:24.147604 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:24.148368 31063 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:24.104364  4673 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.161350 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:24.160938  4790 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:26:24.159971 31694 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 11.
I20260709 12:26:24.180280 31136 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:24.185299 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.183202 31106 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:24.188083 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 43877.9us
I20260709 12:26:24.199350 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:24.200384  5004 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:24.200757  5004 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:24.201655  5004 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.205415 31698 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
W20260709 12:26:24.206250 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.187132  4991 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:24.214394 31695 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:24.164279  5036 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:26:24.218784  5004 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:24.219350  4673 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:24.219900  5033 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.221083  5100 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:24.221802  5100 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:24.170675  5031 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:26:24.222151  5033 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:24.217784  4991 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:24.234300  4991 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:24.215930  5017 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:24.238811  5098 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:26:24.227939 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.242377 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.242660 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.191098  5030 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:24.249954 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:24.250630 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:24.254786 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:24.257283 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.191542  4980 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:26:24.264678 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:24.265398 31063 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:24.266324  4921 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:24.273216 31063 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:24.278334 31136 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:24.279235  5107 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:24.279670  5107 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:24.280535  5107 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.281860  5107 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:24.282142 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.282733  5089 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.283315  5089 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.285475  5089 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:24.287138  5099 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:26:24.295972 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54637.9us
I20260709 12:26:24.296401 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:24.304821 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 121441us
I20260709 12:26:24.310482 31695 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:24.313848 31063 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:26:24.314254 31063 pending_rounds.cc:85] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 9
I20260709 12:26:24.314471 31063 pending_rounds.cc:107] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:26:24.314680 31063 raft_consensus.cc:2889] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:24.315222 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 115948us
W20260709 12:26:24.317401 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 221485us
I20260709 12:26:24.322515  5100 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:24.323951 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:24.323364  5100 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:24.335551 30991 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:24.249336  4942 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.313594  5038 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:24.356580  4962 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.356948  5038 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:24.358093 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.358325 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:24.358850 31695 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:24.359232  5095 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:24.359758  5095 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:24.359828 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:24.360637 30985 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:24.360738  5095 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:24.362370 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 43628.9us
I20260709 12:26:24.313993  4980 consensus_queue.cc:1243] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:24.366300 31135 raft_consensus.cc:2368] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15.
I20260709 12:26:24.344988 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:24.348999 31140 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:24.349095 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:24.335906  5100 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:24.334764  5048 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:26:24.371438 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:24.377820 30990 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 14: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:24.378830 30984 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:26:24.381557 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:24.377051 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:24.319568  5102 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:26:24.382813  4949 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:24.337668  4952 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:26:24.401674 31106 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:24.402822 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.365507  4918 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.369652  4991 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.413640  5036 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:24.397631  5060 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:24.421708 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:24.423282  4952 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:24.423664  4952 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:24.424180 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.424847 31695 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:24.424580  4952 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.425945  4952 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.430305  4949 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.428318 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:24.431301 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.431404 31698 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:24.380700  5105 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:26:24.364692  5038 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.435446 31694 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:24.436802 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:24.438511 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:26:24.441300 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 64351us
I20260709 12:26:24.442905 31137 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:24.446584 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.432624  5093 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:24.454618 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.455449 31698 raft_consensus.cc:2368] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:26:24.434929  4949 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:24.472184 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 90769.8us
W20260709 12:26:24.482595 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.486995 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:24.488443 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 46289.3us
I20260709 12:26:24.492303  5094 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:24.492659 30958 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:24.493186  5116 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:24.494457 31136 pending_rounds.cc:85] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 8
I20260709 12:26:24.494767 31136 pending_rounds.cc:107] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:26:24.494786 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:24.495142 31136 raft_consensus.cc:2889] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:24.491466 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57926.7us
I20260709 12:26:24.497161 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:24.499408 31140 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:24.470100  4942 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:24.472154  4949 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:24.446803  5036 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.552541  4969 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:24.555877 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.556300  5116 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:24.559621  5118 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:24.560405 31695 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:24.558171 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 62438.6us
I20260709 12:26:24.488497  5038 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:24.564450 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 125990us
I20260709 12:26:24.495499  4991 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:24.569074 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:24.555528  4980 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:24.495582  5111 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:24.570719 31138 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:26:24.562022 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 129096us
I20260709 12:26:24.579865  5124 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:24.579928  5123 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:24.558314  4918 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:24.557797  5110 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:24.570614  4949 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:24.458563  5100 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.584152 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.589896  5113 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:26:24.583307 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:24.597412 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.599045 31698 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:24.603950 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:24.604524 30983 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:24.580770  5105 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:24.622004  5105 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.622347  5109 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:26:24.624111  5036 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:24.626832 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47538.3us
I20260709 12:26:24.586151  5124 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:24.633363  5128 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:26:24.634363  5108 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:26:24.634433  5116 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:24.642288 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59054us
I20260709 12:26:24.643239 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.604346  5110 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:24.651556 31698 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:24.652539 31691 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:26:24.653184 30955 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:24.654603  5100 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.656868 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:24.657861 31063 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:24.627436  5123 raft_consensus.cc:2764] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:24.659091  5131 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:26:24.679252 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:24.640467  5105 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:24.691205 31059 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:24.650802  5118 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:24.692232 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:24.697633 30983 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:24.660282  5128 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:24.702576 31060 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:24.681787  4969 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:24.705256 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.607224  5111 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:24.716518 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 88361.3us
W20260709 12:26:24.718657 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 65566.8us
I20260709 12:26:24.730713  5118 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.702756  5126 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:26:24.729183  5033 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:24.725638  5110 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.725193  5139 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.738713 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:24.708192  5109 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:24.752384 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.752997 31698 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:24.754720 30955 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.744400  4942 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:24.762185 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:24.769274 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.747820  5033 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:24.780192  5033 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.781495  5033 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:24.715947  5060 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:26:24.776111 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.786958 31698 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:24.778390 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:24.788002 31060 raft_consensus.cc:606] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.788456 31060 raft_consensus.cc:614] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:24.778620  5124 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.778811  5128 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.779244  5111 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:24.757550  5109 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:24.733047  4949 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.800504 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:24.802647 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.757544  5123 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:26:24.765928  5102 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:26:24.789448  5135 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:24.790416  5126 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:24.850869 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.852393  5109 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.853665  5109 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:24.859087  5128 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:24.782655  4969 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.830071  4942 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.867981 31060 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:26:24.864092 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 113782us
I20260709 12:26:24.798238  5113 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:26:24.844835  5036 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:24.884433  5138 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:26:24.901629 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.902204 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.902519 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.902887 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.865247  5123 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:24.905890 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 62948.9us
W20260709 12:26:24.916442 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 164077us
I20260709 12:26:24.919288 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:24.923864 31140 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:24.885470  5131 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:24.932330 31144 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:24.932633 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.916630  5126 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.937358 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 180214us
I20260709 12:26:24.962513  5126 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.937767 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.892750  5135 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:24.942902  4962 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.002s
I20260709 12:26:24.947615 31143 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.949616 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:24.965852 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.966899 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:24.967393 31140 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:24.950783 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:24.970487  5130 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:24.971171  5130 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:24.952271 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:24.954704 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:24.955726  5138 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.976152  5138 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.976435 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:24.978471  5138 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:24.975515 31141 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:24.975495 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.980312  5095 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:24.975703  5036 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.980901  5036 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.982599 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.983165  5036 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:24.986332  5095 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:24.989156 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:24.989725 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:24.992045  5102 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:24.992509  5095 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:24.992664  5102 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:24.995433  5102 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:24.996027  5102 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:24.996899  5102 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:24.989764 31695 raft_consensus.cc:606] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:24.999468 31695 raft_consensus.cc:614] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:25.003448 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:25.004884 31695 raft_consensus.cc:606] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.010179 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.011143  5095 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:25.012594 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 48530us
I20260709 12:26:25.028745 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:24.933848  5030 consensus_meta.cc:287] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.668s	user 0.011s	sys 0.000s
I20260709 12:26:25.037485 31109 leader_election.cc:400] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 13 pre-election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:26:25.038319 31058 raft_consensus.cc:2410] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:26:25.030700  4980 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:25.043691  4980 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.047576 31140 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
W20260709 12:26:25.047978 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.048636  5102 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.055289 31059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:25.057518 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.059417 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.063531 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:24.937927  5123 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:25.065506 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 139301us
I20260709 12:26:25.039253  5030 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14.
I20260709 12:26:25.117076  5146 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.146874  5139 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.105222  4991 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.123s
I20260709 12:26:25.128088  5135 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.149752  5107 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:26:25.139844 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 76374.7us
W20260709 12:26:25.119115 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 196490us
W20260709 12:26:25.117638  5004 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.898s	user 0.012s	sys 0.000s
I20260709 12:26:25.213528 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:25.056174 31059 raft_consensus.cc:606] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.214078 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.215721 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:25.216123 31135 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 12:26:25.125497  5038 consensus_meta.cc:287] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.001s
I20260709 12:26:25.217685  5038 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:25.067263 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:25.220021 31062 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:25.220366  5107 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:25.105295 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.160962 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.135459 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.222041  5107 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.152640 31143 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:25.122922  5126 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.151050  4942 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.092375  4969 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.148978  5105 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.225541  4969 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.228134  5105 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.155319 31138 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:26:25.225102  5107 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.222332 31142 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:25.224864  5126 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:26:25.216051 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:25.230981 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.200735 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.200214 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:25.235706 31694 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:25.240511 31140 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:25.061136  4918 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:25.132066  4952 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.702s	user 0.006s	sys 0.007s
I20260709 12:26:25.246402 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.244901 31033 leader_election.cc:400] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15.
I20260709 12:26:25.255717 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.265748 31033 leader_election.cc:403] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:25.262705 31139 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:25.269006 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:25.260274 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:25.269737 31695 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:25.270907 31141 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.086709  5142 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:25.270438 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.274782 31137 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
W20260709 12:26:25.212220  5137 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.430s	user 0.001s	sys 0.000s
I20260709 12:26:25.276070 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.277685 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.228717  5124 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:25.211978  5123 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:25.281868 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 65875.7us
I20260709 12:26:25.287529 30985 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:25.283928 30983 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:25.277753 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:25.286015 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:25.288851 31698 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 13.
I20260709 12:26:25.290915 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:25.290133 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:25.296517 30985 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:25.298269 31692 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:26:25.298511 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 73698.9us
I20260709 12:26:25.299050 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:25.300235 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.301244 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.306790 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:25.307004 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 76085.7us
I20260709 12:26:25.279117  5128 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:25.301486 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.280704  5087 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:25.313680 31696 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:25.313939 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.314209 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:25.322324 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.252836  4779 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:26:25.329399  4779 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:25.329771  4779 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:25.330361 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:25.330863 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 117454us
I20260709 12:26:25.330615  4779 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.331702 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:25.331856  4779 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.291821  4952 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:25.295995  5126 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:25.324316  4942 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.333986  5126 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.339434  5126 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:25.322643  5138 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:26:25.313233  5141 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.322387  5123 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:26:25.334000 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.349677 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:25.350508 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.314715  5128 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:25.325722 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:25.354519 31692 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.324318 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.356251 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.357381 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.328943 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.358884 30991 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:25.359603 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:25.322113  4969 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.360031 30991 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:25.328027 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:25.362710 30984 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 21: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:25.311908 31109 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:25.363749 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51903.7us
I20260709 12:26:25.363862  5100 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:25.364641  5100 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:25.365517  5100 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.358465  5128 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.368491  5128 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.370338  5119 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:25.377851 30986 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:25.382936 30983 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:25.305035 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:26:25.384972 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 80009.1us
I20260709 12:26:25.387962 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.387832  4969 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:25.388674  4969 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:25.389039  4969 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:25.403653 31141 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:25.331171 30990 raft_consensus.cc:2393] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:25.358048 30992 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:25.412480  5141 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:25.416160 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.420037  5060 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:25.421370  5123 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:25.422055  5123 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.409706 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:25.424912  5123 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.424986  5161 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:25.425845  5161 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:25.428536 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 96919.1us
I20260709 12:26:25.429443 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.430423  5133 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:25.431422  5133 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:25.434183  5138 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:25.434593  5138 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:25.435472  5138 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.436704  5138 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.427166  5161 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.438944  5144 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.439903  5144 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.441295 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.442696  5133 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:25.442950  5144 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.443111  5133 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:25.443532  5133 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.429561 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.445421  5144 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:25.446965 31136 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:25.447533  4942 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:25.448314 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2940) took 1102 ms. Trace:
I20260709 12:26:25.448931 30987 rpcz_store.cc:276] 0709 12:26:24.345692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.345822 (+   130us) service_pool.cc:224] Handling call
0709 12:26:24.361528 (+ 15706us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:24.361894 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.361923 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:24.362559 (+   636us) log.cc:844] Serialized 52 byte log entry
0709 12:26:24.393607 (+ 31048us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:25.204284 (+810677us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b1800495460. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf 00007f2eeb8ae038
0709 12:26:25.278134 (+ 73850us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.447850 (+169716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.447926 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.447965 (+    39us) raft_consensus.cc:1685] finished
0709 12:26:25.448068 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.448227 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169404,"prepare.queue_time_us":5861,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":38352,"spinlock_wait_cycles":16865792,"thread_start_us":104369,"threads_started":2,"wal-append.queue_time_us":116624}
I20260709 12:26:25.438562 31692 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
W20260709 12:26:25.463431 30876 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2439e7934b98446ca4e7d82381e8a499 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e9cccf7eb2a545ef8c2169b67ff261bf is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:25.458895 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.498340 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:25.510306 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.502954 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:25.511456 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.518440  4952 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 14. Current term is 15.
I20260709 12:26:25.520210 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.522544 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.496768  5146 raft_consensus.cc:2764] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:26:25.530494  5106 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:25.542531 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 133215us
I20260709 12:26:25.502120  4892 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:25.408862  5126 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.546407 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:25.520117  5123 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:25.523370  5144 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:25.559852 30987 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.561756 31135 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.544872  5105 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:25.569525 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:25.551286  5107 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:25.567814 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:25.552906  5142 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.550818  5106 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:25.575649 31696 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.550710  4892 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:25.580797 31692 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:25.582019 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.554809  4952 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.591250 30983 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:25.570571  5004 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.581773  5124 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:25.580289  4673 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:25.584605  5142 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:25.605441 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 111354us
I20260709 12:26:25.586829  5106 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.594564  4952 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.610579  4673 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.610020  5107 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:25.598439  5133 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.610889  5142 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.613629  4952 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.610993  5094 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:25.632710  5141 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.633194  5141 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:25.604308  5004 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.635239  5004 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.613253  5124 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:25.618191  4942 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:25.656921 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.627115  4949 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:25.657786 30990 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:25.654309  5004 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:25.627199  4892 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.649291  5107 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.677160  5147 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:26:25.659857  4942 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.632745  5094 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.719916  5094 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.722280  4673 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.724970 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:25.675196  5167 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:26:25.659375  4969 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.666103  4949 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:25.683211  5163 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:26:25.698480  5126 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.703994  5124 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.706431  5107 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.713896  4952 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.717567  5141 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:25.723099  5168 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:26:25.729748 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 159793us
I20260709 12:26:25.731212 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.732537 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:25.733116 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:25.670961  5004 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.737459 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:25.726294  5105 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.741344 31058 raft_consensus.cc:2410] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:25.742961 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.750774  5111 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:25.751798  5123 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:25.752457  5123 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.746699  5105 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.747097 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:25.754267  5105 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.754626  5112 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:25.754747  5105 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.755179  5112 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 21 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:25.755266  5105 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.756129  5112 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.747509  5093 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:25.747610  5090 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:25.748812  5133 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.749838 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:25.761094 31059 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:25.740226 31065 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:25.739869 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.744196  4949 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.752621  5106 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.772421  4969 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.735030  5100 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:26:25.774756 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.775463  4892 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.775382 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.776168 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:25.777293 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.777861 31144 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:25.778088 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.778759 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.779337 31142 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:25.760195  5093 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:25.780894  5093 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.782035  5093 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:25.785796  5004 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.760730  5094 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:25.787361  5094 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:25.787726  5094 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.791152 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.792515  5144 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:25.795416 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:25.795536 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:25.798534  5091 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:25.798718 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58929.5us
I20260709 12:26:25.799623 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.800354 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:25.800974 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:25.801240 31059 raft_consensus.cc:2410] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:26:25.803282 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:25.803597  5091 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:25.804124  4942 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.804312  5091 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.804849  5091 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:25.805161  5091 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:25.807619  5165 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.219s	user 0.004s	sys 0.000s
I20260709 12:26:25.809403 31063 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:25.810530 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:25.811712 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:25.813597 30987 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:25.816309 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:25.816563 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 198381us
I20260709 12:26:25.817502 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:25.822753 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:25.824398 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.817340  5147 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:25.827903 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.846367  5147 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.846747  5146 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.846931  4980 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:25.847287  4991 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.847255  5146 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.847875 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:25.829682 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.834167  5106 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.834944  4949 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.836074 31058 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:25.836555  5176 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:25.837126 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.862646 31143 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
W20260709 12:26:25.837635 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.838179  4952 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:25.839155  5174 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:25.839648 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:25.843683  5114 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:25.844215  4779 raft_consensus.cc:993] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:25.852538 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:25.855592 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:25.856107 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:25.846779  5147 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:25.870779 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:25.894536 31698 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:25.850788  5090 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:25.823272  5105 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:25.872215 31058 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 21. Ops: []
I20260709 12:26:25.904718 31063 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:25.818349  5123 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.885149 30992 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:25.890008 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:25.862190  4949 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.870931 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:25.898411 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:25.899668  4779 raft_consensus.cc:1081] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:25.892984  5144 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:25.923566 31059 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.905615 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.875386 31144 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:26:25.925287 31144 pending_rounds.cc:85] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 10
I20260709 12:26:25.925551 31144 pending_rounds.cc:107] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:26:25.925781 31144 raft_consensus.cc:2889] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:25.877010  5048 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.892546  5095 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:25.912762 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:25.916009  5105 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:25.916635  5004 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:25.919931 30991 raft_consensus.cc:2368] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 21. Current term is 22.
I20260709 12:26:25.935959  5094 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.848135  4991 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:26:25.890676  5114 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:25.922461 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:25.922734 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:25.902480  5126 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.938105 31067 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:25.938237  5126 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.938376  5114 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.928848  5124 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:26:25.940383  5126 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.943107  5123 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:25.945322 31062 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:25.919754  4949 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.946415  5088 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.947099  5088 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.949728  5088 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:25.950379 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 79679.3us
I20260709 12:26:25.951190  4991 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:25.951246 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.951843  4991 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:25.929396  5144 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.952098  4918 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:25.957729 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 128149us
I20260709 12:26:25.961578 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:25.957847 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 94390.8us
I20260709 12:26:25.960007 31109 leader_election.cc:400] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 21. Current term is 22.
I20260709 12:26:25.967432 31109 leader_election.cc:403] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:26:25.954536 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49023.2us
I20260709 12:26:25.968518 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:25.920106  5161 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:26.007237 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 47361.7us
I20260709 12:26:25.955466  4991 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:26.008638  5178 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:26.009225 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:26.010411 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.011169  5060 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:26.011950 31106 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:26.013036  4985 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:26.014514 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.014612  5033 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:26.015236 30989 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:26.015309  4815 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:26.017781 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.017865 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:26.019238 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:25.954073  5144 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:26.021076 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 68649.8us
I20260709 12:26:26.018378 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:26.022495 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:26.023327 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:26.024446 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:26.026914 31144 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:26.029232 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.029579 30986 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.957342  5038 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:26.023599 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:25.958091  5115 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:25.956951  5114 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:25.969877  5095 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:26.037451 31694 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:26.045192 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:26.031270 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:26.038767 31692 raft_consensus.cc:2393] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:25.970458  5094 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:25.970988  4892 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.041127 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.034294 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:26.058753 31696 raft_consensus.cc:2393] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:26.042361 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:26.034780 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.057734 31698 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:25.921356  5142 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:25.958029  5110 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:26:26.061841  5146 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:26.062059  5121 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 12:26:26.030607  4942 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:26.064497  5091 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.069054  5095 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.075034 30987 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:25.930843  5048 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.094013 31141 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:26:26.035312  5087 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.101715 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.102174 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:26.102421 30989 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:26.040740  4673 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:26.038368 31144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:42006
I20260709 12:26:26.108342 31144 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:26.108738 31144 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:26.099964 31141 pending_rounds.cc:85] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 11
I20260709 12:26:26.114822 31141 pending_rounds.cc:107] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:26:26.115103 31141 raft_consensus.cc:2889] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:26.095068 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 63873.4us
I20260709 12:26:26.042438  5100 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:26.061766  5142 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:26.040593  5106 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:26.043394  5112 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.040022  5161 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:26.058172  4991 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:26.044116  4876 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:26.140427 31143 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:26.141429 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:26.123041 31144 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:26.141919 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 174781us
I20260709 12:26:26.099251  5030 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:26.160046 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:26.160693 30991 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:26.161335 31141 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:26.161446 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.162549 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:26.162998 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:26.163359 31144 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:26.120761  5141 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:26.094842  4892 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.123907  5106 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:26.178515 31137 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:26.148068  4876 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:26.184336 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 139202us
I20260709 12:26:26.147931  5100 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:26.185041 31106 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:26.203549 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 284714us
I20260709 12:26:26.203946 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:26.166129  5091 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:26.204310 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.209192 31106 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:26.122390  5087 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.182114  5106 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:26.221756 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 59299.6us
I20260709 12:26:26.186569  4876 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:26.149557  5112 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.159976  4991 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.237103 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.238088 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:26.210333 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.219625  5175 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.220273  5088 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:26.186417  4980 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:26:26.221716  5030 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:26.257944 31142 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.221949  5141 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:26:26.296579  5142 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:26.298166  5048 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:26.304134  4949 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.234347  5161 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.222204  5111 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
W20260709 12:26:26.294401  5174 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:26.207422  5152 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
W20260709 12:26:26.311667 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 101400us
I20260709 12:26:26.315488  5090 raft_consensus.cc:2764] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:26:26.315824  5187 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:26:26.340677 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 136436us
I20260709 12:26:26.341578 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.320221  5030 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:26.342182 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:26.342942 31062 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:26.331446  5141 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:26.345594 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 121782us
W20260709 12:26:26.314061  5182 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:26.333431  4991 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:26.311203 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 74181.8us
I20260709 12:26:26.208108  5100 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.275034  5175 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.331921  5004 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.346405 30955 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.350044 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.350751 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.363191 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:26.363708 31138 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:26.352945 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:26.364629 31144 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.355031 31061 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:26.360621 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.362438 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.367234 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.367285 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:26.363998 31065 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:26.376541 31142 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:26.355963  5126 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:26.354413  5115 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:26.361323  5105 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:26.384657 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:26.389204 31694 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:26:26.390355 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44058.6us
I20260709 12:26:26.391076 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.392266 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.392757 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:26.393328 31689 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:26.396828 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:26.375751  5030 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:26.399135  4892 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:26.399821 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:26.400506 31696 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:26.385777 31144 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:26.385864 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:26.382802  5126 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.405941 31692 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:26.383244  4949 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:26.384473  5048 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:26.404402 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.394649  5090 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.372102  5095 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:26:26.372991  5004 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.373806  4991 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:26:26.399807  4985 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:26.411149 31106 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.396034  5128 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.413002 31692 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:26.423966 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:26.424774 31061 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:26.397500  5142 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:26.361310  5141 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.430406 31062 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:26.408840  5126 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:26.409545  5111 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:26.431521 30991 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:26.443482  5038 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:26.395538  5088 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:26:26.455739 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:26.456329 31065 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.414583  4673 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:26.476176 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 75087.9us
I20260709 12:26:26.426403  4892 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.482450 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:26.431238  5142 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:26.451545  5087 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:26.460132  4673 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:26.503027 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.440285  5128 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.451073  5091 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:26:26.414683  5090 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.490446  5095 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:26.508518 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.509670 30955 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.495359  5142 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.502365  5106 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.513577 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.490732  4892 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.516595  4892 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:26.516917  5106 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:26.517184  5119 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:26.517552  5119 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:26.517875  5186 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:26.518298  5186 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:26.518743  5186 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.518410  5119 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.519457  5119 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.505280  4673 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.504784  5087 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:26.510087  5197 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.155s
I20260709 12:26:26.501564  5030 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.511164  5095 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:26.531301  5122 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:26.532543  4980 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:26.531059 30955 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.536379  5087 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:26.512930  5123 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:26.554881  5122 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:26.536856  5087 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:26.556344  5147 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.000s
I20260709 12:26:26.574316 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.575590 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:26.577193 30955 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:26.559634  5100 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:26.568495  5122 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:26.571648  5199 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 12:26:26.572559  5200 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:26.536391  4980 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:26.539615 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:26.587656 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48149.8us
I20260709 12:26:26.588477 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:26.641736 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 53345.3us
I20260709 12:26:26.584143  5122 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:26.642693  5206 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:26.579787  5109 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:26.581390  5152 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:26.589622  5175 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:26.589547  5004 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:26.652436 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 249879us
W20260709 12:26:26.653894 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 201181us
I20260709 12:26:26.653932 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.592304  5123 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:26.587556  4980 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:26.669631 31689 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:26.671919 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:26.579852  5147 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.677698 30983 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:26.674945 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.674516 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.698524 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.702597 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.702732 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.704774 31136 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:26.707147 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:26.707156 31696 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:26.709576 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.710420 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:26.712687 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.713277 31136 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:26.711103  5185 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:26.711452  5206 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:26:26.715507 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.654739  5119 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:26.716116 30990 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:26.656607  5141 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:26.716876 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:26.717412 30989 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:26.656155  5109 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:26.726373 30884 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:26.727456  5105 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:26.728081  5038 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:26.670776  5197 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.670382  5004 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:26.711755  5107 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:26.729431  5188 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:26.589962  5095 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:26.749349 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 172224us
I20260709 12:26:26.707206  5175 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:26:26.736701  5109 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:26.734200  5111 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:26.764935 31065 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:26.741896  5119 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:26.771678 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.745474  5202 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:26.764827  5188 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:26.778579 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 69118.8us
I20260709 12:26:26.778585 30884 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:26.782893 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.748692  4949 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
W20260709 12:26:26.767148  5060 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.756s	user 0.005s	sys 0.006s
I20260709 12:26:26.741302  5038 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.767555  5128 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:26.788436 31065 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:26.789384 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:26.789978 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:26.787354 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:26.796100 31060 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:26:26.796446 31060 pending_rounds.cc:85] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 10
I20260709 12:26:26.796674 31060 pending_rounds.cc:107] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:26:26.796933 31060 raft_consensus.cc:2889] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.797246 31060 pending_rounds.cc:107] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:26:26.797488 31060 raft_consensus.cc:2889] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.799130 31063 pending_rounds.cc:85] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 14
I20260709 12:26:26.799405 31063 pending_rounds.cc:107] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:26:26.799635 31063 raft_consensus.cc:2889] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.783517  5188 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:26.781466  5206 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:26.775511  5111 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.781857  5100 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:26.801702 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.817160 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.753337  5197 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.766816  5214 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:26:26.804997 30884 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:26:26.813743 30882 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 322) took 12524 ms (client timeout 15000 ms). Trace:
I20260709 12:26:26.793051  5060 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 21. Current term is 22.
I20260709 12:26:26.817952 31141 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
W20260709 12:26:26.802781  4876 consensus_meta.cc:287] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.002s
I20260709 12:26:26.801972  5123 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:26.823935 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.833992  5206 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.834604  5206 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.837340  5206 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:26.842460  5210 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:26:26.819388  5105 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:26.797142  5215 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 12:26:26.858067 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 70778.1us
I20260709 12:26:26.859293 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:26.826622 30884 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 8 to 10, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:26.811408  5107 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:26:26.868213 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.868783 31136 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:26.844234  5004 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:26.851780  5105 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.874420  5111 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:26.874815  4918 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.875521  5038 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:26.880344  5112 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:26.883587  5147 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:26.886853  5126 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.820263  5100 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.870092  5106 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:26.895460  4985 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:26.896055 31139 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
W20260709 12:26:26.896755  5211 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:26:26.898396 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:26.900000  5213 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 12:26:26.903115 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.873102 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.963272 31138 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:26.849238  5123 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:26.855491  5161 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:26.825879  4876 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:26.917392 31140 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:26.949483 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.950879 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:26.978727 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:26.916663  5105 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:26.955753 31696 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:26.985255 31696 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:26.971534 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:26.921646  5048 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.507s	user 0.001s	sys 0.008s
I20260709 12:26:26.916098  4892 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:26:26.864012  5060 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.893821  4673 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.984561 30884 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 8 to 10, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:26.965394 31060 raft_consensus.cc:1240] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:26:26.902899  5004 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.990939 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.993767  5139 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:26:26.985407  5217 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.022967 31696 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:26.996687 31144 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:26.994983 31140 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:26.985054  5215 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:26.990057  4985 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:26.992384  5048 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:26.861271  5119 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.025249  5214 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:27.058415  5214 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.058982  5214 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.053241 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:27.062736 31140 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:27.028074  5004 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:26.975521  4980 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.027639  5147 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.031514  5030 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.039896  5175 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:27.077601 31140 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:27.080986 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:26.983659  4918 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:26.820024 30882 rpcz_store.cc:269] 0709 12:26:14.238988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.278366 (+ 39378us) service_pool.cc:224] Handling call
0709 12:26:26.754273 (+12475907us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:19.005229 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:19.059364 (+ 54135us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:19.066519 (+  7155us) write_op.cc:435] Acquired schema lock
0709 12:26:19.072074 (+  5555us) tablet.cc:662] Decoding operations
0709 12:26:19.680604 (+608530us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:19.680663 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:19.680691 (+    28us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:26:19.689960 (+  9269us) tablet.cc:701] Row locks acquired
0709 12:26:19.689979 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:26:19.690122 (+   143us) write_op.cc:270] Start()
0709 12:26:19.690269 (+   147us) write_op.cc:276] Timestamp: P: 1783599979690078 usec, L: 0
0709 12:26:19.690295 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:26:19.691348 (+  1053us) log.cc:844] Serialized 11236 byte log entry
0709 12:26:21.046145 (+1354797us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:22.408806 (+1362661us) op_driver.cc:464] REPLICATION: finished
0709 12:26:22.832636 (+423830us) write_op.cc:301] APPLY: starting
0709 12:26:22.856443 (+ 23807us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:23.071706 (+215263us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:23.071731 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:23.472980 (+401249us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:23.473550 (+   570us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=58,mrs_lookups=0
0709 12:26:23.473614 (+    64us) write_op.cc:312] APPLY: finished
0709 12:26:23.493364 (+ 19750us) log.cc:844] Serialized 252 byte log entry
0709 12:26:23.701641 (+208277us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:23.735551 (+ 33910us) write_op.cc:454] Released schema lock
0709 12:26:23.735652 (+   101us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:23.736245 (+   593us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:23.736278 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26320,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":365316,"apply.run_cpu_time_us":93933,"apply.run_wall_time_us":931049,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125405,"num_ops":29,"prepare.queue_time_us":381743,"prepare.run_cpu_time_us":66882,"prepare.run_wall_time_us":1158112,"raft.queue_time_us":227591,"raft.run_cpu_time_us":26797,"raft.run_wall_time_us":215626,"replication_time_us":2718296,"spinlock_wait_cycles":226105600,"thread_start_us":1123452,"threads_started":5,"wal-append.queue_time_us":875527,"wal-append.run_cpu_time_us":62688,"wal-append.run_wall_time_us":4086765}]]}
W20260709 12:26:27.084255  5184 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.029s	sys 0.032s
I20260709 12:26:26.975145  5106 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.094230 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.068303 30884 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:27.102802  4952 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.103436  4952 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.106052  4952 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.107246 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:27.092911  5184 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.029s	sys 0.032s
I20260709 12:26:27.083069  5175 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.027496  5060 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.119182 31140 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:27.092901  5144 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:27.071259  5110 consensus_queue.cc:1059] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
W20260709 12:26:27.122013  5087 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.004s
I20260709 12:26:27.124662  5175 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:27.052361  5217 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.081734  5147 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.088285  5030 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.187866  5048 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:27.076335  5123 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.262600 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:27.228562  5161 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.232966  5095 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.264423 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:27.193212  5206 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.248332  5048 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:27.188021  5144 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:27.133285  5197 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.253072  5119 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.352427 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.346805 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.355499 31138 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:27.357187 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:27.257395 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:27.248673  5128 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:27.273538  4876 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:27.321959  5207 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.329154  5142 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.785s	user 0.006s	sys 0.001s
I20260709 12:26:27.329008  5048 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.330049  4918 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.330492  4673 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.331192  4980 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.332479  5030 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:27.333412  5226 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 12:26:27.399999 31060 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:26:27.366017  5095 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:27.275161  5126 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:27.336087  5229 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:26:27.337340  5217 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:27.337925  5222 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 12:26:27.252259  5193 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.274s
W20260709 12:26:27.339098  5228 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
W20260709 12:26:27.339876  5224 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
W20260709 12:26:27.340054  5122 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.681s	user 0.006s	sys 0.003s
W20260709 12:26:27.340531  5230 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:26:27.343185  5147 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.376459  5142 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.344333  5060 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.194666  5087 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.364070 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.425160 31136 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
W20260709 12:26:27.346165  5186 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.827s	user 0.006s	sys 0.002s
W20260709 12:26:27.372207 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 114892us
I20260709 12:26:27.369539  5090 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:27.373114 31064 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:27.374349 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:27.363050  5109 consensus_meta.cc:287] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.002s
I20260709 12:26:27.351405  4952 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:27.400427 31060 pending_rounds.cc:85] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 10
I20260709 12:26:27.431167 31060 pending_rounds.cc:107] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:26:27.431537 31060 raft_consensus.cc:2889] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:27.272742  5161 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.401871 31068 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:27.326072  5214 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.435266  5214 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.430994  5147 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:27.436368  5147 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:27.428013  5188 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.613s	user 0.006s	sys 0.003s
I20260709 12:26:27.439051  5188 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.443810  5161 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.393527  4876 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:27.444558 31689 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:27.447369 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:27.447937 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.448580 31136 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:27.449397 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:27.422087  5122 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.452680  5147 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.363613  5119 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.453961  5122 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.454700  5119 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.365839  5128 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.427618  5186 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.406661  5048 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:27.343315  5106 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.408236  4980 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:27.372619 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:27.426679 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:27.471460 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44885.7us
I20260709 12:26:27.427129 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:27.451866  5175 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.456547  5128 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.457325  4952 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.458036  5152 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.458827 31698 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:27.459803  5087 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:27.459939 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.504231 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:27.447978 31694 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
W20260709 12:26:27.461030  5234 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:27.461411  5119 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:27.462762 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2810) took 1610 ms. Trace:
I20260709 12:26:27.463337  4942 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:27.405774  5095 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.320961  5206 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.468241 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:27.473644 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 46580.9us
I20260709 12:26:27.455916  5109 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.466043  5100 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.475262 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.476492  5105 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.476961 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:27.477137 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:27.481117  5188 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.537778 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:27.542515 31691 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:27.542546 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:27.542755  5188 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.481528  5093 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:27.372213  5207 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.484095  5186 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.486083 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2710) took 1135 ms. Trace:
I20260709 12:26:27.526852  5095 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.486953  5142 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.486889  4980 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.497031  5220 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:27.501596 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:27.506376 31142 rpcz_store.cc:276] 0709 12:26:25.852186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.852324 (+   138us) service_pool.cc:224] Handling call
0709 12:26:27.462584 (+1610260us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b540029e0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:27.462677 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843791104}
I20260709 12:26:27.469349  5185 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:27.554677 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:27.528913 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:27.531947 31138 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:27.537164 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:27.538864 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:27.543680  5188 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.540776 31068 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:27.538255 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:27.538851 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:27.522300  5128 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.425897  5144 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.508137  4942 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:27.527318  5215 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:27.489282  5175 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.525871  5147 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:27.492470 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:27.581096 31137 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:27.551446 30992 rpcz_store.cc:276] 0709 12:26:26.350597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.351572 (+   975us) service_pool.cc:224] Handling call
0709 12:26:26.351865 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.352216 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:26.352245 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.356345 (+  4100us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.902199 (+545854us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b18003aa8c0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:26:27.025143 (+122944us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:27.025181 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.025240 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.469747 (+444507us) raft_consensus.cc:1685] finished
0709 12:26:27.469774 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.486024 (+ 16250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4833,"raft.queue_time_us":2257,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":10698,"spinlock_wait_cycles":39322496,"thread_start_us":122747,"threads_started":1,"wal-append.queue_time_us":722500}
I20260709 12:26:27.527643  5123 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:27.563865 31695 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:27.481897  5202 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:27.498987  5152 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.574653 31060 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:26:27.525141  5119 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.529784  5033 consensus_queue.cc:1059] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:26:27.576280 31693 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:27.578056 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:27.553483  5093 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:27.558720  5220 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:27.578028  5186 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:27.576179  5110 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:27.585007  5202 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.577227  5128 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:27.686888 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 158188us
I20260709 12:26:27.584664  5147 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.607183  5186 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.615283  5113 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:27.591516  5123 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.575799  5185 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.671463  4876 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:27.620126  5202 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.620499  5248 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:27.732038 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:27.619815  5060 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:27.739494 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:27.767277 31065 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:27.646553  5087 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.688930  4673 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:27.768420  4673 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.769150 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:27.770260 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:27.770686  4673 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:27.770823 31058 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:27.771411 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:27.771836 31058 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:27.779385 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:27.780122  5161 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:27.780601  5161 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:27.780977  5161 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.782706  4991 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:27.785290  5224 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.334s	user 0.001s	sys 0.006s
I20260709 12:26:27.788960 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:27.792162  5105 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.792569  5161 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.794622  5161 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:27.795594 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2815) took 1093 ms. Trace:
W20260709 12:26:27.795739 31139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2664) took 1655 ms (client timeout 2197 ms). Trace:
W20260709 12:26:27.796190 31139 rpcz_store.cc:269] 0709 12:26:26.140284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.140524 (+   240us) service_pool.cc:224] Handling call
0709 12:26:27.795584 (+1655060us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54001e91d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:27.795681 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527595904}
I20260709 12:26:27.796278 31058 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:27.799800 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:27.800508 31059 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:27.753060 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:27.801800 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48950.8us
I20260709 12:26:27.804070  5128 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.804984  5217 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:27.805379  5217 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:27.806252  5217 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.807245  5217 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:27.619343  5110 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.810303  5110 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.764135 31142 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 20. Current term is 21. Ops: [18.15-20.16]
I20260709 12:26:27.735342  5175 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.811655  5175 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.813787  5175 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.814314 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:27.795993 31144 rpcz_store.cc:276] 0709 12:26:26.702122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.702357 (+   235us) service_pool.cc:224] Handling call
0709 12:26:27.795404 (+1093047us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b540006f4d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:27.795536 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674557824}
I20260709 12:26:27.751488  5186 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.817862 31062 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.818006 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.823616 31142 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:27.738564  5030 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.739220  5121 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.609436  4942 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.810014 31059 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:27.824782 31106 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:27.751070  5111 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:27.751154  5113 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:27.762638  5257 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:26:27.821156 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 243202us
I20260709 12:26:27.756299  4952 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.766610  5126 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.762408  5188 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.760042  5123 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.874536 30991 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:27.833529 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 136323us
I20260709 12:26:27.833801 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:27.876271 31059 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:27.756567  5109 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.880112 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.881093 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:27.840041 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:27.822077  5091 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.758646  5185 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.830489  4673 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:27.892411  4673 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:27.893332  4673 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.894008 31693 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:27.894551 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:27.895370 31695 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:27.757143  4876 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.838166  4991 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
W20260709 12:26:27.847913  5250 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:26:27.855365  5249 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:26:27.855913  5214 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.856037  5244 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:26:27.833962  5121 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.857630 30983 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:27.857837  5207 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.830813  5030 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.858820  5119 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:27.859140 30990 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:27.859165 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:27.860023 30987 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:27.861685 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:27.828820  5248 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:27.863993 31065 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:27.867771 31064 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:27.754285  5220 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.870270 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:27.878664 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:27.883069 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:27.840463  5106 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:27.884905  4980 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.840605  4942 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.838682  5111 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.885876  5147 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.824445  5060 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.863649  5141 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:27.891502  5091 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.896400 31067 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:27.897634 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:27.896792 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:27.900257 30988 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:27.840579  5206 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:27.838405  5257 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:27.824961  5087 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.930589 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:27.856860  5245 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:26:27.938759 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:27.821327  5093 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:27.868290 31062 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:27.874392  5126 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.490507  5142 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.569844  5142 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.570241  5197 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:28.570924 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.570871  5197 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:28.571838 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 748444us
I20260709 12:26:28.572933  5197 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:28.284332  4876 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:28.575351  4673 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:27.964274 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:28.575917  4673 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.576275 31693 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:28.577523  5111 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:27.996559  5113 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:27.951172 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:28.578061  4673 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:28.580101  5197 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:28.580365 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:28.580715  5197 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:28.208022 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 310529us
I20260709 12:26:28.581864 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:28.582095 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:28.069871  5258 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:28.583309 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 686541us
I20260709 12:26:28.583585  5258 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.584188 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:28.585175  5248 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:28.155036 31137 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.573s	user 0.007s	sys 0.003s
W20260709 12:26:28.018926  5256 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:28.030270  5186 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.586879 31137 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
W20260709 12:26:28.008081  5250 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:28.587350 31138 raft_consensus.cc:2393] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:28.588181 31058 raft_consensus.cc:1240] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:28.589356 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:28.023090  5229 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:27.939348 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:28.590020 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.584146 30991 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:28.582712 31064 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:28.575675  5142 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:28.590739 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:28.591523 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:28.592281 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:28.398581  4991 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:28.591513 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:28.077782 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:28.685657 30987 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:28.686556 31139 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:27.963871 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:28.578898  5113 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:28.611682 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:28.585829  5258 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:28.068840  5095 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 12:26:28.742689  5095 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.743853 31692 raft_consensus.cc:2368] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18.
I20260709 12:26:28.340731  5122 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:28.621701  4980 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:26:28.630170 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:28.308344  5048 consensus_meta.cc:287] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.822s	user 0.006s	sys 0.005s
I20260709 12:26:28.247995  5185 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.595208 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:28.754664 31059 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
W20260709 12:26:27.934170  5004 consensus_meta.cc:287] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.851s	user 0.008s	sys 0.002s
W20260709 12:26:28.685787 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 608058us
W20260709 12:26:28.173321 31109 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.996s (SENT) [suppressed 36 similar messages]
I20260709 12:26:28.621024 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 1255) took 1129 ms. Trace:
W20260709 12:26:28.538759  5144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.008s	sys 0.007s
I20260709 12:26:28.687111 31142 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:28.763445 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:28.765106  5121 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:28.766286  5091 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:28.585829  5060 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:28.587469  5141 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.620074  5114 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:28.580850  5087 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:28.688417 31697 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:28.687901 31067 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:28.344568  5215 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:28.777858 31067 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:28.713887 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:27.950798  5214 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.779668 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1695) took 1274 ms. Trace:
I20260709 12:26:28.781152  5214 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:28.720058 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:28.582814  5197 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:28.586841  5106 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:28.363737  5253 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:28.626245  5119 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:28.621964  4876 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:28.723088 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:28.621727  5111 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.656507  5206 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.715680 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:28.721244  4991 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:28.732156 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.731513  5030 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:28.719602  5267 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:28.744737 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:28.879061 31138 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:28.720057  5245 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:26:28.884514 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1361) took 2497 ms. Trace:
I20260709 12:26:28.748133 30991 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:28.886487 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2983) took 1008 ms. Trace:
I20260709 12:26:28.796571  5114 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:28.884994 31690 rpcz_store.cc:276] 0709 12:26:26.387017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.387164 (+   147us) service_pool.cc:224] Handling call
0709 12:26:26.387407 (+   243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.588393 (+200986us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:26.650233 (+ 61840us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.444991 (+794758us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b1800443440. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:26:27.576432 (+131441us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.586036 (+  9604us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.586251 (+   215us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:27.586301 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.586389 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.884150 (+1297761us) raft_consensus.cc:1685] finished
0709 12:26:28.884274 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.884438 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32742,"prepare.queue_time_us":57134,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":22906,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":20272,"spinlock_wait_cycles":8803584,"thread_start_us":306196,"threads_started":2,"wal-append.queue_time_us":921381}
I20260709 12:26:28.193626  5202 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.888450  5114 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:28.888762  5202 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:28.746062 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:28.889490 30988 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:28.758534  5185 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:28.895015  5263 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:28.895921  5185 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:28.896423 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3103) took 1128 ms. Trace:
I20260709 12:26:28.761914 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:28.772148 31142 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:28.744896  5095 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:28.773834 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 189732us
I20260709 12:26:28.775069 31059 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:28.778453 30987 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:28.897017 31063 rpcz_store.cc:276] 0709 12:26:27.768420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.773180 (+  4760us) service_pool.cc:224] Handling call
0709 12:26:27.773486 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.773848 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.773873 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.774313 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 12:26:28.215109 (+440796us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:28.215171 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.215272 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.896176 (+680904us) raft_consensus.cc:1685] finished
0709 12:26:28.896204 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.896352 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14133,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"thread_start_us":440377,"threads_started":1,"wal-append.queue_time_us":971952}
I20260709 12:26:28.898808 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1701) took 1083 ms. Trace:
I20260709 12:26:28.898936 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:28.899166 31059 rpcz_store.cc:276] 0709 12:26:27.815417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.763082 (+947665us) service_pool.cc:224] Handling call
0709 12:26:28.774654 (+ 11572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.898578 (+123924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.898760 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:28.899559 30985 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:28.900408 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:28.901674 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:28.763463  5144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.008s	sys 0.013s
I20260709 12:26:28.905345 30987 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:28.780103 31061 rpcz_store.cc:276] 0709 12:26:27.500130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.500629 (+   499us) service_pool.cc:224] Handling call
0709 12:26:27.500999 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.509144 (+  8145us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:27.509190 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.540560 (+ 31370us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.732875 (+192315us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:27.732928 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.733008 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.774499 (+1041491us) raft_consensus.cc:1685] finished
0709 12:26:28.774520 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.774653 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7703,"raft.queue_time_us":41152,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":165313,"threads_started":1,"wal-append.queue_time_us":282213}
I20260709 12:26:28.908893 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3110) took 1111 ms. Trace:
I20260709 12:26:28.909392 31138 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:28.909310 31058 rpcz_store.cc:276] 0709 12:26:27.797444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.811186 (+ 13742us) service_pool.cc:224] Handling call
0709 12:26:28.587958 (+776772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.650243 (+ 62285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.908703 (+258460us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54002ce2d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:28.908835 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785628544}
I20260709 12:26:28.786970 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:28.796393 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 212813us
I20260709 12:26:28.689215 31060 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 12:26:28.824476 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.940843 30983 raft_consensus.cc:2410] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:26:28.758149  5113 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:28.773180  5207 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:28.775686  5141 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.873075 31697 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:26:28.752996  5122 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.763193  5004 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:28.877279  5111 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:28.886821 30991 rpcz_store.cc:276] 0709 12:26:27.878273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.878435 (+   162us) service_pool.cc:224] Handling call
0709 12:26:28.886429 (+1007994us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:28.750360 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:28.896658 31694 raft_consensus.cc:2368] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:28.759505  5126 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:28.757256 31137 rpcz_store.cc:276] 0709 12:26:27.492052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.492215 (+   163us) service_pool.cc:224] Handling call
0709 12:26:28.620965 (+1128750us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:28.928656  5144 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.002151 30985 raft_consensus.cc:2410] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:28.748008  5257 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:28.911438 31062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 2929) took 1444 ms (client timeout 1915 ms). Trace:
I20260709 12:26:28.913832 31068 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:28.914723 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:28.802992  5215 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.748373  5262 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.147s
I20260709 12:26:28.872263  5253 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.776371  5176 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:28.877154  4876 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:28.919822 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.924399  4991 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:28.877635  5206 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:28.868111  5106 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:28.928779 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:28.932847 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 321212us
I20260709 12:26:28.941322 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2988) took 1001 ms. Trace:
I20260709 12:26:28.966812 31697 pending_rounds.cc:85] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 15
I20260709 12:26:28.973105 30988 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:28.967867 30955 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.025070 30983 rpcz_store.cc:276] 0709 12:26:27.940262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.590882 (+650620us) service_pool.cc:224] Handling call
0709 12:26:28.786788 (+195906us) spinlock_profiling.cc:243] Waited 7.36 ms on lock 0x7b5800103b98. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:26:28.941269 (+154481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16932096}
I20260709 12:26:28.993568 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:28.976792 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 220192us
I20260709 12:26:28.928794  5267 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:28.995244 31136 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
W20260709 12:26:29.025872 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58083.6us
I20260709 12:26:28.781463 31067 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:28.977029  5110 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.004209 31062 rpcz_store.cc:269] 0709 12:26:27.467399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.467973 (+   574us) service_pool.cc:224] Handling call
0709 12:26:28.911235 (+1443262us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54001bb054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:28.911362 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669865344}
I20260709 12:26:29.008602 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 1019) took 1045 ms. Trace:
I20260709 12:26:28.977397  5113 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:29.018381  5265 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.018872 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.019199  5278 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:29.019275 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:29.001818  5274 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.024679 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.021406  5186 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:28.976799  5260 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:26:29.028218 31065 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:29.010565  5144 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:28.975425  5207 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.066874 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.031111  5277 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:26:29.031270  5267 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.032073  5276 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:29.036993 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2934) took 1217 ms. Trace:
I20260709 12:26:29.033577 31693 rpcz_store.cc:276] 0709 12:26:27.962906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.577361 (+614455us) service_pool.cc:224] Handling call
0709 12:26:28.914526 (+337165us) spinlock_profiling.cc:243] Waited 5.41 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:26:29.008519 (+ 93993us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12432640}
I20260709 12:26:29.021190  5176 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.035993 30955 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.034794 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.037758  5030 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.039609 31106 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:29.088503 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48974.9us
I20260709 12:26:29.039832 31691 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:29.089555 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.024552  5115 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.090287 30990 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:29.038743 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:26:28.997176 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 823828us
I20260709 12:26:29.087044  5176 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:29.093724 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57853.5us
I20260709 12:26:29.055335 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2989) took 1104 ms. Trace:
I20260709 12:26:29.055984  5111 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:29.056341  5273 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:29.082316  5126 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:29.058703  5268 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:29.107678 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:29.059034  5271 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.346s	user 0.002s	sys 0.000s
I20260709 12:26:29.064128 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.108865 30992 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:29.105607  5260 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:29.109493  5260 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:29.107661  5268 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:29.110435  5260 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.111657  5268 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.112360  5121 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.104780  5115 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.112943  5121 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.113633 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.114755 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.115017  5121 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.106348  5185 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:29.115761 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.116858 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:29.038269  5110 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.117347 30986 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:29.095944  5114 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:29.118635  5114 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:29.041422  5265 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.084658 31067 rpcz_store.cc:276] 0709 12:26:27.819861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.781020 (+961159us) service_pool.cc:224] Handling call
0709 12:26:28.781278 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.032764 (+251486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.036914 (+  4150us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:29.086311 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 51586.6us
I20260709 12:26:29.117100  5152 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.131546  5265 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.133277 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:29.133848 31138 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.025190 31697 pending_rounds.cc:107] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:26:29.134471 31697 raft_consensus.cc:2889] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:29.135059 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.135653 31136 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:26:29.136603 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.137140  5216 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.137693  5220 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:29.137656  5216 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.041013  5248 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:29.138540 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.048882  5060 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:29.065954  5206 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.142706 31697 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:26:29.094599 30955 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:29.143662 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49126.5us
I20260709 12:26:29.109915 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.144484 31138 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:29.150020 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.151168 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.089188 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.094377 30987 rpcz_store.cc:276] 0709 12:26:27.951272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.905991 (+954719us) service_pool.cc:224] Handling call
0709 12:26:28.906280 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.906476 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:28.906710 (+   234us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.054885 (+148175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.054965 (+    80us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:29.055034 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.055123 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.055265 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19641,"prepare.queue_time_us":111,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":17026,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":81,"spinlock_wait_cycles":161024,"thread_start_us":130804,"threads_started":1,"wal-append.queue_time_us":157160}
I20260709 12:26:29.151877 31062 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:29.097215 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 1013) took 1198 ms. Trace:
W20260709 12:26:29.099126 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 60441.3us
I20260709 12:26:29.078655  5253 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.057338  5004 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.077121  4876 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.082886  5281 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.084635  5276 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.095400  4991 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:29.115860  5185 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.119179 30991 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:29.108340 30990 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:29.076946 31065 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:29.065773 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3107) took 1279 ms. Trace:
I20260709 12:26:29.185463 31060 rpcz_store.cc:276] 0709 12:26:27.786410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.786534 (+   124us) service_pool.cc:224] Handling call
0709 12:26:27.786813 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.940955 (+1154142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.065543 (+124588us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b5400383954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:29.065708 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068365056}
I20260709 12:26:29.112304  4942 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.125752  5246 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.279s	user 0.007s	sys 0.009s
I20260709 12:26:29.049734  5274 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.126457  5114 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.128108  5110 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.129582  4673 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.137607 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.199429 31136 raft_consensus.cc:2410] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:29.200380 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:29.201213 31136 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:29.138038 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:29.133759  5266 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.410s	user 0.001s	sys 0.000s
I20260709 12:26:29.153515 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.212854 31697 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:29.154629 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.040872  5278 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.155596 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:29.155850 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 66737.5us
I20260709 12:26:29.156827 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:29.222616 31690 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:29.223310 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 59402.6us
I20260709 12:26:29.159051 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:29.224174 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 65885.2us
I20260709 12:26:29.225082 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.152843 31691 rpcz_store.cc:276] 0709 12:26:27.891458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.902541 (+ 11083us) service_pool.cc:224] Handling call
0709 12:26:29.039458 (+1136917us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:26:29.039674 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.089762 (+ 50088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.089948 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11012992}
I20260709 12:26:29.187315  5185 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.160786 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.160858  5144 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.161477 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.191082  5122 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.162187  5262 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:29.163256 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.199100  5115 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.238739 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.142740  5112 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.164160 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.169286 30986 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:29.079238  5113 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.155072  5060 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:29.246299  5202 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:29.246898  5202 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:29.093976  5030 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.150750  5095 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.165359  5280 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.155330  5152 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:29.155055  5219 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.202255 31137 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:29.218991  5215 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.194821  5216 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.210825 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 1017) took 1248 ms. Trace:
I20260709 12:26:29.119282  5126 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.180740  4991 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.157156 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.226717 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.233392  5087 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:29.181643  5268 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:29.228179 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.261101 31136 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:29.261468 31695 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:29.229699 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.179899  5276 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:29.162043  5186 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:29.233103 30958 connection.cc:256] RPC call timeout handler was delayed by -1.051700620000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
I20260709 12:26:29.233731 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.180636  5281 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:29.236018 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 115777us
I20260709 12:26:29.237762 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.140237  5248 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.267750 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.268114  5248 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.268913 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:29.269624 30987 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:29.270373 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.242048 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.223949  5093 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.245115  5112 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.246438  5060 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.195971 31141 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:29.232600  5258 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.249383 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.274720  5216 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:29.275079 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.275517  5216 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.276059  5216 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:29.160194 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.250909 31698 rpcz_store.cc:276] 0709 12:26:27.962289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.964407 (+  2118us) service_pool.cc:224] Handling call
0709 12:26:28.784259 (+819852us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef086dfc0 00007f2ef0858929 00007f2ef08583d2 00007f2ee9dcfbd4
0709 12:26:28.784516 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:28.785003 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:29.007854 (+222851us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.105904 (+ 98050us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.108441 (+  2537us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.108582 (+   141us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:29.108614 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.108670 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.210452 (+101782us) raft_consensus.cc:1685] finished
0709 12:26:29.210574 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.210756 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50220,"prepare.queue_time_us":247175,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":1361,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":20327,"spinlock_wait_cycles":13277952,"thread_start_us":305097,"threads_started":2,"wal-append.queue_time_us":143396}
I20260709 12:26:29.259080 31690 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:29.235452  5197 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:29.277876 31067 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:29.260149 31691 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:29.278158  5197 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:29.278637  5161 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:29.279080  5197 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.282104 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2667) took 3111 ms. Trace:
I20260709 12:26:29.248188  5095 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.265161 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.275444 31137 raft_consensus.cc:2410] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:26:29.193629  4942 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.282522 31143 rpcz_store.cc:276] 0709 12:26:26.153200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.153366 (+   166us) service_pool.cc:224] Handling call
0709 12:26:26.153629 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.166589 (+ 12960us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.166618 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.853443 (+686825us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b18004e4f60. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87752a6 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a
0709 12:26:27.450502 (+597059us) spinlock_profiling.cc:243] Waited 996 us on lock 0x7b2c000e06f8. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:26:27.450964 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.562070 (+111106us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:27.562107 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.562167 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.125211 (+1563044us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:29.264682 (+139471us) raft_consensus.cc:1685] finished
0709 12:26:29.264713 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.264889 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15643,"raft.queue_time_us":12999,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":9325,"spinlock_wait_cycles":44393472,"thread_start_us":110752,"threads_started":1,"wal-append.queue_time_us":285716}
I20260709 12:26:29.213963  4673 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.289902 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.293663 31135 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:29.284296 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.262609 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:29.295122 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.289611 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.296548 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.260560  4991 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.282611  5274 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.300230 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.307164 30992 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:29.300966 31697 raft_consensus.cc:2410] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:26:29.247864  5202 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.305888 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.263499  5110 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.309895 31696 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:29.283077  5186 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.319998 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.283288  5293 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.322422 31143 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:29.319453 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:29.326359 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 101426us
I20260709 12:26:29.326812 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.327852 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.328418  5274 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:29.328833 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.328933  5274 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.329599 31690 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:29.286146  5207 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.298801  5152 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:29.331661 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:29.341015 31697 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:29.331027  5274 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.344765 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.347433 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:29.248443  5280 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.346715 31697 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:29.313901  5267 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.343801 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.290589  5265 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:26:29.388476  5126 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:29.391088 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 52067.5us
I20260709 12:26:29.343267  5262 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.389964  4673 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:29.376995 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 81966.4us
I20260709 12:26:29.292510  5093 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.263221  5219 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.349053 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:29.366747 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.450321 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.303682  5121 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:26:29.299077  5111 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:26:29.335968  5112 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.491025 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.491853 31061 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:29.292351  5258 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.333104  5186 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:29.403559 31696 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:29.348176  5207 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:29.443244 30987 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:29.371675  5004 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:29.333256  5110 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.323699  5290 consensus_queue.cc:1059] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 12:26:29.451023  5278 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.451419  5294 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.453095  5289 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:26:29.462428  5152 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.463313  5095 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:29.339815  5293 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.530859  5294 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.532634  5293 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:29.475984 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 126967us
I20260709 12:26:29.533815 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.448786  5301 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:29.480924  5205 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:29.473270  5115 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:26:29.484961 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.526819  5004 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:29.535916 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51036us
I20260709 12:26:29.536084  5004 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.536648 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.537709 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:29.485183 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.483959  5280 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:29.497910 31696 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1354) took 2599 ms (client timeout 1996 ms). Trace:
I20260709 12:26:29.500262 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:29.449200  4876 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:29.510758  5121 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:29.512184  4942 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.483338  5248 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:29.521358 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.492493  5206 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:29.497270  5060 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:29.534430  5106 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.506719  5215 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:29.449331  5267 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.477567  5141 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.467278  5262 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.538028  5300 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:29.527001  5260 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:29.543167 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.565507  5308 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:29.565742  5111 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:29.567860  5219 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.566104 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.541297  5278 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:29.570086  5119 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:29.572307 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.533744  5294 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:29.573472 31142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2822) took 1703 ms (client timeout 2190 ms). Trace:
I20260709 12:26:29.498189  5176 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.647156  5119 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:29.647616  5106 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.648372  5176 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.648070  5119 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.574990  5281 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:29.576287 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3115) took 1002 ms. Trace:
W20260709 12:26:29.582361 31692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 1010) took 1683 ms (client timeout 1936 ms). Trace:
I20260709 12:26:29.649644  5119 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.640276  5111 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:29.650708  5111 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.649323 31063 rpcz_store.cc:276] 0709 12:26:28.574079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.899110 (+325031us) service_pool.cc:224] Handling call
0709 12:26:28.899390 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.092393 (+193003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.092427 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.092466 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:29.092495 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.576064 (+483569us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54000c6554. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:29.576107 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.576231 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443473792}
W20260709 12:26:29.564965 31696 rpcz_store.cc:269] 0709 12:26:26.898002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.898121 (+   119us) service_pool.cc:224] Handling call
0709 12:26:27.778559 (+880438us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b18001f4ac0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8cf646 00007f2eeb80958e 00007f2eeb88af9e 00007f2eeb88c7de 00007f2eeb89827c 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c
0709 12:26:29.309582 (+1531023us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b580090c2d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb7f00aa 00007f2eeb872447 00007f2eeb88afe7 00007f2eeb88c7de 00007f2eeb89827c 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c
0709 12:26:29.497832 (+188250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3518220032}
W20260709 12:26:29.654595 31065 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 1706) took 1838 ms (client timeout 2105 ms). Trace:
W20260709 12:26:29.601285 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 101149us
I20260709 12:26:29.598898  5206 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.648399 31144 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:29.615981 31062 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
W20260709 12:26:29.655020 31065 rpcz_store.cc:269] 0709 12:26:27.816602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.589146 (+772544us) service_pool.cc:224] Handling call
0709 12:26:29.654440 (+1065294us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54002a3d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:29.654540 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868026752}
I20260709 12:26:29.556006  5205 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.612633 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.663265 30990 raft_consensus.cc:2410] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 12:26:29.641110 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 98839.5us
I20260709 12:26:29.610914 31690 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:29.642477  5223 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:29.666353  5223 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:29.666756  5223 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.667302  5205 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.667382 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58004 (request call id 7) took 1698 ms. Trace:
I20260709 12:26:29.667927 30989 rpcz_store.cc:276] 0709 12:26:27.969044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.929901 (+960857us) service_pool.cc:224] Handling call
0709 12:26:29.667122 (+737221us) spinlock_profiling.cc:243] Waited 51 ms on lock 0x7b54002c1d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb89a735 00007f2eeb8929fa 00007f2ef085a4c9 00007f2ee9dd6c54 00007f2ee9dd6be2 00007f2ee9dd6b11 00007f2ee9dd6a8c 00007f2ee9dd5d42 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:29.667293 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117309312}
I20260709 12:26:29.555776  5144 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:26:29.643683 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 77643.7us
I20260709 12:26:29.670528  5144 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:29.671275  5248 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.671456 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:29.654959  5245 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:29.612856  4991 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.673039  5276 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.671357  5144 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:29.644843 31142 rpcz_store.cc:269] 0709 12:26:27.869873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.870039 (+   166us) service_pool.cc:224] Handling call
0709 12:26:29.573297 (+1703258us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54000942d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:29.573413 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231606528}
W20260709 12:26:29.674738 31694 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1357) took 2139 ms (client timeout 1690 ms). Trace:
I20260709 12:26:29.675386  5276 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.674999  5278 raft_consensus.cc:2764] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:26:29.675783 30876 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2439e7934b98446ca4e7d82381e8a499 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e9cccf7eb2a545ef8c2169b67ff261bf is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:29.676097 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3101) took 1923 ms. Trace:
I20260709 12:26:29.657003  5110 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.676442 31068 rpcz_store.cc:276] 0709 12:26:27.752773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.755897 (+  3124us) service_pool.cc:224] Handling call
0709 12:26:28.913573 (+1157676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.009863 (+ 96290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.675921 (+666058us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001bb050. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:29.676045 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659975552}
I20260709 12:26:29.644886 30986 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:29.679006 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:29.673074 31693 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:29.539059  5185 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:29.649834 31692 rpcz_store.cc:269] 0709 12:26:27.890539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.929627 (+ 39088us) service_pool.cc:224] Handling call
0709 12:26:29.574019 (+1644392us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54006a09d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:29.574155 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355606912}
I20260709 12:26:29.584040 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.530866  5265 raft_consensus.cc:3055] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:29.601135 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:29.682201 31689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 1014) took 1731 ms (client timeout 2205 ms). Trace:
W20260709 12:26:29.682709 31689 rpcz_store.cc:269] 0709 12:26:27.951076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.002464 (+ 51388us) service_pool.cc:224] Handling call
0709 12:26:29.600928 (+1598464us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b5800011498. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee876fa22 00007f2ef16888d9 00007f2ef07e2ee0 00007f2ef08ac0ab 00007f2ef08ac131 00007f2ef085a0a0 00007f2ef0859984 00007f2ee9dd46d4 00007f2ee9dd4662
0709 12:26:29.682127 (+ 81199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14638464}
I20260709 12:26:29.658063 30984 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:29.609140  5258 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.695832 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.605094  5275 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.696499 31137 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:26:29.616492  4980 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:29.555593  5121 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:29.656464  5253 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.669929 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:29.701009 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.701313 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.666016  5220 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:26:29.702555 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.535212  5030 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:29.679593 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.705192 30986 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:29.705988 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.702340 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.707016 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.646116  5060 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:29.675139 31694 rpcz_store.cc:269] 0709 12:26:27.535743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.537971 (+  2228us) service_pool.cc:224] Handling call
0709 12:26:29.674549 (+2136578us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54004a2c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:29.674668 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3058192768}
I20260709 12:26:29.681315 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.687979 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.710173 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.710657 31142 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:29.689894 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.634205  5300 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:29.689913 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.657541  5206 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:29.641232  5215 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.701606 30989 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:29.711661 31065 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:29.712842 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.718745 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.719321 31063 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:29.695271  5265 raft_consensus.cc:740] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:29.720547 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.715521 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.736634 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.736832 30986 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:29.737188 31062 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:29.737825 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:29.605242  5291 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.213s
I20260709 12:26:29.717033 31058 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:29.717588 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.717675 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:29.714617 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.751452 31060 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:29.752759  5060 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:29.753482  5060 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:29.754660  5060 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.756269 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.756503 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.757386 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:29.722033  5141 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.757055 30989 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:29.722599  5262 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.723284 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.768146 31059 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:29.704560  5275 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.727430 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:29.728678 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.770045 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.729619  5267 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.729799 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 1248) took 1452 ms. Trace:
I20260709 12:26:29.695377  4949 consensus_queue.cc:1059] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:26:29.756111  5311 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:29.704469  5253 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.712911  4980 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:29.717432  4876 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.709578  5220 consensus_queue.cc:1243] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:29.699895  5207 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:29.734496 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.795094 31068 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:29.738449 30990 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:29.744598 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.746026 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:29.762077 31060 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:29.764250 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.797219 30986 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:29.765623 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.656286  5260 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.771924 31692 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:29.699083  5278 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:26:29.770802 30989 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:29.733722  5093 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:29.773701 30985 rpcz_store.cc:276] 0709 12:26:28.277732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.900198 (+622466us) service_pool.cc:224] Handling call
0709 12:26:29.729604 (+829406us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54003467d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:29.729740 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232187648}
W20260709 12:26:29.731447 31139 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2823) took 1852 ms (client timeout 2021 ms). Trace:
W20260709 12:26:29.804237 31139 rpcz_store.cc:269] 0709 12:26:27.878931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.879793 (+   862us) service_pool.cc:224] Handling call
0709 12:26:29.731297 (+1851504us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54000947d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:29.731391 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1850297728}
I20260709 12:26:29.777500 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.807027 31137 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:29.710093  5216 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.783365  5311 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:29.799577 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 53674.1us
I20260709 12:26:29.800426 31063 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:26:29.821405 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:29.780298 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 296386us
I20260709 12:26:29.824826 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:29.828523  5093 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:29.829241  5093 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.830086 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.831454 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:29.794148  4980 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.833303 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:29.832199 31694 raft_consensus.cc:2376] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16. Re-sending same reply.
I20260709 12:26:29.837893 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.826387 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.838891 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.798233  5207 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.839583 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.835048  5202 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.820638  5106 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:29.841641 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.842883 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.850270 30986 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:29.843932 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.803793  5141 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:29.844658 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:29.845147 31691 raft_consensus.cc:2368] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:29.847879 31695 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:29.836910 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:29.854406 31693 raft_consensus.cc:2368] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:29.855297 31693 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:29.852106 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:29.857074 31109 leader_election.cc:400] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:29.834569  5311 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:29.843358  4980 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:29.857573 31109 leader_election.cc:403] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:29.846668  5207 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:29.853338  5141 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.863207 31109 leader_election.cc:400] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:29.838120  5267 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:29.857898 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.857152 31068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:29.926569 31068 raft_consensus.cc:606] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:29.926936 31068 raft_consensus.cc:614] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:29.857038 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.927809 30986 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:29.844337  4985 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:29.861140 31059 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:29.922266 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.931258 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:29.932642 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:29.849586  5106 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.798843  5265 consensus_queue.cc:260] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.794921  5300 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:29.921612 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57487.2us
I20260709 12:26:29.775944  5121 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.962566  5215 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.962981  4991 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.926056 30985 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:29.963858 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.943960 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:29.965027 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:29.915104  5308 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:26:29.965517 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.966405 31143 raft_consensus.cc:2410] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:26:29.962627 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.970404 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.971908 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.972648 31067 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:29.973604 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.976691 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.978312 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:29.979971  5317 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:26:29.953425 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.955984 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.861490  5262 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:29.981079 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:29.867053  5141 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:29.981328 31067 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 14: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:29.860890  5109 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:29.982124  5109 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.982448  5093 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.982574  5109 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:29.982750 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:29.983405 31067 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:29.850016  5202 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.984557 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:29.981412 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:29.985121 31067 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.985522 31059 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:29.986413  5202 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.959409  5095 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:29.987262  5095 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:29.987663  5095 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:29.981072  5262 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.990558  5265 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.990679  5262 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:29.991434  5296 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:29.992573  5260 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:29.993214  5300 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:29.993587  5109 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:29.995733  5216 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:29.995847  5109 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.000495  5186 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.002843 31694 raft_consensus.cc:2410] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 22 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:30.006656  5142 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:30.007757  5142 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:30.008105  5142 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.011832  5185 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.015064  5278 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.015552  5278 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.979772 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.017396 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.018301 30985 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:30.019410 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:29.957755 31062 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:26:29.956681 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.020409 31064 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.948952  5281 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:30.021562  5281 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:30.022102 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.022650 30992 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:29.949076  5176 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:30.023365  5262 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:30.024189  5313 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:29.864488  5030 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.026196 31139 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:29.970263  5111 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:29.928270  4942 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.015668 31067 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:30.028973 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.047354 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.029330 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:30.048861 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 203660us
I20260709 12:26:30.049661 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:30.033330 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.037451 30990 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
W20260709 12:26:30.028777 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.052155 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.043745 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.049587 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.029561 31064 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:29.934741  4985 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.052961 30986 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.048646 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.053759 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2723) took 1046 ms. Trace:
I20260709 12:26:30.056383 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.056660 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.057641 31062 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:30.056480 30990 rpcz_store.cc:276] 0709 12:26:29.006880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.698244 (+691364us) service_pool.cc:224] Handling call
0709 12:26:30.053615 (+355371us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.058444 31067 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:30.058588 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.059190 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.056051 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.060482 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:30.055588 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:30.061785 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.059783 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.063366 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.064865 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.065445 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.070609 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.071424 31059 raft_consensus.cc:2368] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:30.071655 31064 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:30.066943 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:30.065732 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.065080 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.043113  5106 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.065825 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:30.068436 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.079686 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.079953 31060 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:30.038475  5267 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:30.017535  5278 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.076941 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.078248 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.092180 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:30.093509 31695 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:30.094408 31065 raft_consensus.cc:2393] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:30.092353  5205 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:26:30.095875 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.097054 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.080997  4991 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:30.097765 30983 raft_consensus.cc:2410] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:26:30.097818  4991 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.098325  4991 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.098760 30992 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:30.008981  5142 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.099682 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.099874  5142 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:30.100823 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.101898 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.102806 31142 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:30.103648 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.073781  5290 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:30.105943 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2834) took 1338 ms. Trace:
I20260709 12:26:30.106312 31140 rpcz_store.cc:276] 0709 12:26:28.767539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.768700 (+  1161us) service_pool.cc:224] Handling call
0709 12:26:28.769084 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.036149 (+267065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.036186 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.036244 (+    58us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:29.036291 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.105734 (+1069443us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b540012cfd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:30.105773 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.105894 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":613681408}
I20260709 12:26:30.063522  4985 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.118876  5253 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.068501  5323 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:30.127511 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.046412  5110 consensus_queue.cc:1059] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:26:30.092554  5308 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:30.054854  5114 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:30.133388 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:30.129458 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.101845 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.061316  5202 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:30.157016 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.143496 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.138348 31137 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:30.161339 31140 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:30.141772 30955 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.154726 30986 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:30.161780 30983 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
W20260709 12:26:30.176797 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 97193.4us
I20260709 12:26:30.180039 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.181178 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.181186 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:30.054801  5093 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.179759 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.062345  4942 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.130189  5121 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.193714 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.130743  5110 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:30.189469 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:30.188278 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.196278 31135 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:30.175904  5221 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:30.093170  4949 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:30.196606 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2716) took 1231 ms. Trace:
I20260709 12:26:30.179658  5277 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:30.191977  5290 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:30.205149 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.223560 30983 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:30.224668 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.225488 30983 raft_consensus.cc:2368] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:30.193866  5323 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
W20260709 12:26:30.206847 30984 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2986) took 2302 ms (client timeout 2234 ms). Trace:
W20260709 12:26:30.141005 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 62846.8us
I20260709 12:26:30.208781 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:30.211087 30992 rpcz_store.cc:276] 0709 12:26:28.964727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.101641 (+1136914us) service_pool.cc:224] Handling call
0709 12:26:30.196546 (+ 94905us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.128824  5311 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.132120  5308 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:30.189998  5328 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:30.194571  5248 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:30.240799 31661 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.208977  5093 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:30.185189 31664 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.197376  5267 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:30.240094 30984 rpcz_store.cc:269] 0709 12:26:27.903993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.904147 (+   154us) service_pool.cc:224] Handling call
0709 12:26:30.206600 (+2302453us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:30.206756 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2161780224}
I20260709 12:26:30.240943 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1244) took 2186 ms. Trace:
I20260709 12:26:30.182090  5202 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.198879  5110 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.081037  5281 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.193796  5048 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:26:30.252600 30992 rpcz_store.cc:276] 0709 12:26:28.054271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.240236 (+2185965us) service_pool.cc:224] Handling call
0709 12:26:30.240494 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.240646 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.240668 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.240703 (+    35us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:30.240740 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.240790 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.240896 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.210168  4980 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:30.267361  5223 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.000s
I20260709 12:26:30.267875  5223 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.269958  5223 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.198484  5253 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:26:30.210595  4949 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:30.315704  4949 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.316215  4985 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.260270  4876 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:30.261636  4942 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:30.245347  5308 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:30.263616  5202 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.221050  5325 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:30.322777  5325 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.323173  5325 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:30.323566  5197 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:30.323704  5202 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:30.324719 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 139613us
I20260709 12:26:30.271168 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2837) took 1112 ms. Trace:
I20260709 12:26:30.325464 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.326181  5202 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:30.326681  5202 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:30.327533  5202 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.271845  4991 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.273921  5186 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:30.201527  5109 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:26:30.278494  5258 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:30.278954  5262 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.279217 30985 raft_consensus.cc:2368] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 20. Current term is 21.
I20260709 12:26:30.207718  5206 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.242715  5221 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:30.279631 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 70916.8us
I20260709 12:26:30.279762  4673 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:30.279884  5328 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:30.280633  5257 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:30.286626  5248 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:30.288108 31144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2830) took 1600 ms (client timeout 1810 ms). Trace:
I20260709 12:26:30.376686 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:30.376768 31144 rpcz_store.cc:269] 0709 12:26:28.687853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.688027 (+   174us) service_pool.cc:224] Handling call
0709 12:26:30.287944 (+1599917us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b54001e8cd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:30.288050 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1797737728}
W20260709 12:26:30.289878  5119 consensus_meta.cc:287] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.634s	user 0.005s	sys 0.003s
I20260709 12:26:30.290428  5030 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:30.382198 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.290112  5110 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.291719  4980 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.295418 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 2995) took 1202 ms. Trace:
I20260709 12:26:30.410502  5308 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.414173 30987 rpcz_store.cc:276] 0709 12:26:29.093220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.270117 (+176897us) service_pool.cc:224] Handling call
0709 12:26:30.295234 (+1025117us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540022d8d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:30.295356 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341794176}
I20260709 12:26:30.299783 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.415874 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.256762 31109 leader_election.cc:400] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:30.416574 31109 leader_election.cc:403] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:30.301340 31068 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:30.418497  5308 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.418697 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.419514 31144 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:30.420482 31143 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 14.
I20260709 12:26:30.420538 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:30.301826 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 61093us
I20260709 12:26:30.422127 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:30.423152  5106 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.423530 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:30.424232 31140 raft_consensus.cc:2368] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22.
I20260709 12:26:30.300428  5275 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.424710 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.425427 31143 raft_consensus.cc:2393] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:30.258085  5111 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.239883  5277 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.426112 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.426375  5106 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:30.427233  5207 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.002s
I20260709 12:26:30.426908  5277 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.427690  5207 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.428507  4949 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.331430 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:30.430327 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 99072.1us
I20260709 12:26:30.331925 31142 raft_consensus.cc:1240] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:30.227584  5290 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:30.200029  5260 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.133s
W20260709 12:26:30.434042 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 404752us
I20260709 12:26:30.253445  5093 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.367797 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:30.369587 31689 pending_rounds.cc:85] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 14
W20260709 12:26:30.435174 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 48691.2us
I20260709 12:26:30.435492 31689 pending_rounds.cc:107] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:26:30.435781 31689 raft_consensus.cc:2889] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.370335 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.265905  5095 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.334678  5176 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:30.383086 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.383412 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:30.450196 31062 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:30.386725 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:30.451023 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.452031 31106 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:30.388582  5248 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.390697 31693 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:30.457686 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:30.373920  5328 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:30.460587  5328 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:30.461287 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.462698 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.463384  5114 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:30.335028  5258 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:30.464020  5114 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.464200 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:30.464962  5114 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:30.464742  5258 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.465348 31106 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:30.465833  5258 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.466080  5114 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:30.267303  5141 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.472136  5141 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.335239  5267 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.473634 30986 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:30.392025  5276 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:30.401046 30989 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:30.401700 31060 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:30.403173  4991 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:30.404829  4985 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.480162 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.480911 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1243) took 2444 ms. Trace:
I20260709 12:26:30.481388 30986 rpcz_store.cc:276] 0709 12:26:28.036850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.197250 (+2160400us) service_pool.cc:224] Handling call
0709 12:26:30.221933 (+ 24683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.474098 (+252165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.480621 (+  6523us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540014ebd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:30.480840 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578331008}
I20260709 12:26:30.406741 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:30.423982 31661 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.303948 30990 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:30.427134 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.325810 31136 rpcz_store.cc:276] 0709 12:26:29.158337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.295202 (+136865us) service_pool.cc:224] Handling call
0709 12:26:29.295533 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.478976 (+183443us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:29.692496 (+213520us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.033364 (+340868us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.033946 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.034128 (+   182us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:30.034175 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.034272 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.270965 (+236693us) raft_consensus.cc:1685] finished
0709 12:26:30.270988 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.271110 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51553,"prepare.queue_time_us":35689,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":134450,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":32389,"thread_start_us":661540,"threads_started":3,"wal-append.queue_time_us":515051}
W20260709 12:26:30.483507 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.432940 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 176299us
I20260709 12:26:30.443974 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:30.446637 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.484881 31142 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 16: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:30.450259 31696 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.486438 31697 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:30.369345  4673 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:30.387760 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.443528  5093 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:30.454664  5260 consensus_queue.cc:1243] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:30.495185 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.453650 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.456873 31692 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:30.458456  5100 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:30.459002  5291 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:30.459050  5048 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:30.510296 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.459609  5278 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:30.510838 31063 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:30.400369  5311 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.513548 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.476377 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.477324 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:30.429900  5207 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.479740  5033 raft_consensus.cc:2764] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:26:30.475023 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.434098  5290 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:30.434388  5122 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:30.511829  4985 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:26:30.487243 31109 leader_election.cc:400] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 20. Current term is 21.
I20260709 12:26:30.533607 31141 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:30.539412 31109 leader_election.cc:403] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:30.539675 30990 raft_consensus.cc:1240] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 12:26:30.539916 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 52813.8us
I20260709 12:26:30.540302 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2718) took 1563 ms. Trace:
I20260709 12:26:30.540553 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.540649 30990 rpcz_store.cc:276] 0709 12:26:28.976476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.537602 (+1561126us) service_pool.cc:224] Handling call
0709 12:26:30.537968 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.540061 (+  2093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.540257 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.487720 31143 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:30.512499  5275 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:30.567867 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.491667 31058 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.445148  5186 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.388067 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:30.492317 31140 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:30.494956 31136 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:30.516230  5048 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.570505 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:30.504125 31106 leader_election.cc:400] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:30.570940  5262 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:30.571411 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 67420.1us
I20260709 12:26:30.571476  5262 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.496263 30985 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:30.572190 31106 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:30.572824 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.515154 30989 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:30.573379 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:30.573812 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1253) took 2294 ms. Trace:
I20260709 12:26:30.477340 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.574203 30989 rpcz_store.cc:276] 0709 12:26:28.279549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.514628 (+2235079us) service_pool.cc:224] Handling call
0709 12:26:30.514948 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.573583 (+ 58635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.573760 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.574728 31137 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:30.517640 30987 raft_consensus.cc:2376] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22. Re-sending same reply.
I20260709 12:26:30.576077 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.573390 30985 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.576848 31060 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:30.575160 30989 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:30.578351  5265 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:30.578536 30989 raft_consensus.cc:606] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.578826  5265 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:30.579306  5265 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:30.579867  5275 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:30.580336  5275 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:30.526140  4980 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:30.582772 31698 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 19: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:30.518553 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:30.476953  5276 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.527192 31068 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.529245 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.479425 31693 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:30.475262  5300 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:26:30.597031 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:30.599804 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.500388  4991 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:30.532521  5122 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.543123 31143 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:26:30.609730 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:30.490093 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 66194.2us
I20260709 12:26:30.554370  5121 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:26:30.555905  5110 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:30.616369 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:30.617051 31143 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.617810 30986 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:30.557200  5095 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.558444  5113 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:30.562356  4949 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:30.562601  5206 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.563169 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.509910  5325 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:30.566696 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.571903  5048 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.588791 31059 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:30.584854 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.591754  5100 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:30.594651 31060 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:30.632251 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.632877 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.605059 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.633378 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.607121 31062 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
W20260709 12:26:30.634773 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.635038 31062 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:30.635417 30955 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.636260 30987 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:30.607805 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.609025 31696 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:30.611289 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.421941 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.639096 31144 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:30.603957 30990 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:30.639870 31063 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:30.640543 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 2997) took 1536 ms. Trace:
I20260709 12:26:30.640910 30990 rpcz_store.cc:276] 0709 12:26:29.103727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.566267 (+1462540us) service_pool.cc:224] Handling call
0709 12:26:30.603746 (+ 37479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.640265 (+ 36519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.640494 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.646489 31058 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:30.647535 31063 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:30.649183 30986 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:30.650265 31661 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.655726 31691 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:26:30.656162 31062 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 20. Current term is 21. Ops: [20.17-20.17]
W20260709 12:26:30.658810 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 296035us
I20260709 12:26:30.660058 31067 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:30.686906 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2734) took 1526 ms. Trace:
W20260709 12:26:30.680353 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 161904us
I20260709 12:26:30.687814 31061 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:30.680135 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44787.2us
I20260709 12:26:30.688476 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.688989 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.619905  5113 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.631582  5267 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:30.722018 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.721585 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.666904 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.706401 31141 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:30.709934 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:30.732867 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 66046.9us
I20260709 12:26:30.733800 31143 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:30.690618 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.735786 30987 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.739467 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:30.741336 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.742697 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.510241  5221 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.637012  5142 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:30.784384 31143 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:30.687389 30989 rpcz_store.cc:276] 0709 12:26:29.160644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.579446 (+1418802us) service_pool.cc:224] Handling call
0709 12:26:30.603013 (+ 23567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.656766 (+ 53753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.656793 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.656833 (+    40us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:30.656868 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.656925 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.686746 (+ 29821us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b54002c1350. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:30.686852 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53377,"raft.queue_time_us":59814,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":11738,"spinlock_wait_cycles":53561600}
I20260709 12:26:30.785913 31143 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:30.669782 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.782652 31067 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 12:26:30.787093 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 117417us
I20260709 12:26:30.786022 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.788763 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.661891  5197 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:30.615520  4673 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.795905 31143 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:30.793455 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:30.687727  5216 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:30.690630  5152 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:30.699144  4980 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:30.800761 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 112360us
I20260709 12:26:30.815948  5291 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:30.816529 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.813674 31143 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:30.818106  4985 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:30.814314  5152 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:30.818908  5152 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.814334  5216 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:30.820063  5216 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.802568  5095 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:30.822667  5095 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:30.792354  5206 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:30.803555  5262 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.814476  5205 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.790306  5260 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:30.818126 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.825567  5260 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.826279  4985 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:30.826534  5260 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.805012 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.829351 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.823212 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:30.830551  4985 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:30.804191  4673 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.817463  5291 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:30.691193  5114 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.111s
I20260709 12:26:30.831841 30989 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:30.831826  5291 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.833766  5325 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:30.833698  5291 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.835362  5291 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:30.803246  5267 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:26:30.778378  5113 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:30.838447 30988 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:30.839395  5109 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.839864  5109 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.819916 30987 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:30.690709  5048 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:30.844476 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.851429 30989 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:30.852382 30958 leader_election.cc:400] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22.
I20260709 12:26:30.853086 30958 leader_election.cc:403] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:30.854678 30985 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:30.856070 30987 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:30.857486 30987 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:26:30.857661 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.655534  5300 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.188s
I20260709 12:26:30.863353 30984 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:30.784121  5142 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:30.860826  4980 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.865587  5033 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22.
I20260709 12:26:30.866155  5048 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:30.686862  4876 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.166s
I20260709 12:26:30.855866  5119 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:30.687021  5176 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.179s
I20260709 12:26:30.945930 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.737084  5060 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.128s
I20260709 12:26:30.722049  5281 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.130s
W20260709 12:26:30.887955 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 145354us
I20260709 12:26:30.881349 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1252) took 2602 ms. Trace:
I20260709 12:26:30.948534 30991 rpcz_store.cc:276] 0709 12:26:28.279251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.286118 (+2006867us) service_pool.cc:224] Handling call
0709 12:26:30.286489 (+   371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.286827 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.286860 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:30.287450 (+   590us) log.cc:844] Serialized 52 byte log entry
0709 12:26:30.778641 (+491191us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:30.778698 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.778859 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.881121 (+102262us) raft_consensus.cc:1685] finished
0709 12:26:30.881139 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.881280 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1305,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":490790,"threads_started":1,"wal-append.queue_time_us":582914}
I20260709 12:26:30.903936 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:30.905342 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:30.914443 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1245) took 2735 ms. Trace:
I20260709 12:26:30.950774 30984 rpcz_store.cc:276] 0709 12:26:28.179162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.302579 (+2123417us) service_pool.cc:224] Handling call
0709 12:26:30.302862 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.889653 (+586791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.914122 (+ 24469us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b540029a4d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:30.914363 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1288581504}
I20260709 12:26:30.856029  5215 raft_consensus.cc:2764] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 12:26:30.801185  5197 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.868759 31067 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:30.918707 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:30.940833 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 152142us
I20260709 12:26:30.942349 30986 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:30.945855 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:30.699255  5122 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.947834 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:30.902330  5142 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:30.960655 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.967136 31696 raft_consensus.cc:2368] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22.
W20260709 12:26:30.967979 30988 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2739) took 1275 ms (client timeout 1664 ms). Trace:
I20260709 12:26:30.873634  5262 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
W20260709 12:26:30.964298 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 306524us
W20260709 12:26:30.968397 30988 rpcz_store.cc:269] 0709 12:26:29.686824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.718546 (+ 31722us) service_pool.cc:224] Handling call
0709 12:26:30.962126 (+1243580us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54000230d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:30.962233 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2573631232}
W20260709 12:26:30.967437 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 309558us
I20260709 12:26:30.870160  5152 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:30.975504 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:30.979884 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.984638 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:30.947506  5329 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:30.986645  5329 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:30.986698 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:30.987565 31065 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.987881  5329 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:30.989377  5329 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.990995 31059 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:26:30.947906  5325 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.964862  5197 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.998317 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:30.994820 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:31.001552 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:30.952739  5030 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:26:31.006949 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.000267 31664 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:30.981829  5300 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:31.008849  5121 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 20. Current term is 21.
I20260709 12:26:30.998137  5205 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:30.998183  5207 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:26:30.985966  5186 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18.
I20260709 12:26:31.007449  5144 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:31.022323  5248 raft_consensus.cc:2764] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:26:31.023789  5100 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:31.027889 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:30.996923  4985 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:31.004129  5344 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:31.035804 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:31.005748  5281 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:31.015115 31065 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:31.051333 31136 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:31.013274  5300 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:30.995385  5060 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:26:31.039903  5206 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:26:31.054548 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.057222 31068 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
W20260709 12:26:31.032503  5330 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.299s	user 0.002s	sys 0.006s
I20260709 12:26:31.020237  4942 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:31.020048  5205 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:31.084385  5205 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.085913  5205 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.086577 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 1379) took 1020 ms. Trace:
I20260709 12:26:31.086959 31694 rpcz_store.cc:276] 0709 12:26:30.066328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.066458 (+   130us) service_pool.cc:224] Handling call
0709 12:26:30.066748 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.074395 (+  7647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.074428 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.074868 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.501558 (+426690us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:30.501596 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.501671 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.009842 (+508171us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:31.086294 (+ 76452us) raft_consensus.cc:1685] finished
0709 12:26:31.086315 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.086519 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7230,"raft.queue_time_us":8107,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":15785,"spinlock_wait_cycles":114275712,"thread_start_us":426425,"threads_started":1,"wal-append.queue_time_us":580169}
I20260709 12:26:31.017570  5087 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.088693  5087 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.091213  5087 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:31.041693  5121 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.093336  5121 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.095752  5121 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:31.076293 31137 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:31.059160  5144 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:31.074668  5354 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:26:31.082795  5353 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:31.082558  5300 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.062331  5281 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:31.061269  5344 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:31.005914  5048 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.003468  4980 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.065284  5186 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:31.077486  5206 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:31.158100 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.162843 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.151597  5281 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:31.149104  5353 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:31.190562  5344 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.191962  5344 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:30.979614  5277 consensus_queue.cc:1175] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce27a72886964e4287059a1d66bd5b6b" responder_term: 20 status { last_received { term: 17 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:26:31.195047  5110 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.573s	user 0.006s	sys 0.002s
W20260709 12:26:31.197398  5328 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.736s	user 0.007s	sys 0.002s
I20260709 12:26:31.198858  5144 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.178151  4942 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:31.200337 31139 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:31.200692  4942 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:31.145706  5276 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.205940  5276 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:31.211272  4991 consensus_meta.cc:287] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.598s	user 0.003s	sys 0.010s
I20260709 12:26:31.212041  4991 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:31.212502  5265 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.000s
I20260709 12:26:31.212953  5265 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.178333  5114 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:26:31.178915  5300 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:31.216882  5113 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:31.218888  5346 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:26:31.219934 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.187189  5354 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:31.220585  5354 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:31.183679  5290 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.002s
I20260709 12:26:31.233170 30988 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:31.222388  5353 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:31.232823 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:31.197788  5328 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:31.237622  5093 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.721s	user 0.006s	sys 0.003s
W20260709 12:26:31.237778  5275 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.652s	user 0.008s	sys 0.001s
I20260709 12:26:31.187534  5186 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:31.221422  5354 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.249974 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1247) took 2972 ms. Trace:
I20260709 12:26:31.190562  5206 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.237690  4673 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:26:31.245086  5152 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:31.258819  5349 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:26:31.264530  5033 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:31.238113  5290 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.249696  5093 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.275187  5354 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:31.296250 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 392364us
I20260709 12:26:31.262617  5275 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.278973  5206 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:31.304481 31142 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:31.305227 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:31.305924 31060 raft_consensus.cc:2368] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19.
I20260709 12:26:31.306489 31068 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:31.302469 31065 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:31.289361  5113 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:31.309212 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 307801us
W20260709 12:26:31.310379 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 310191us
I20260709 12:26:31.312426 31140 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:31.313982 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.314553 31141 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:31.293551 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1246) took 3024 ms. Trace:
I20260709 12:26:31.315708 31059 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:31.316236 30983 rpcz_store.cc:276] 0709 12:26:28.269563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.335932 (+2066369us) service_pool.cc:224] Handling call
0709 12:26:30.336314 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.336545 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:30.858900 (+522355us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.293184 (+434284us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.293245 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:31.293314 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.293380 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.293497 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7046,"prepare.queue_time_us":656727,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":715,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4528,"thread_start_us":916786,"threads_started":2}
I20260709 12:26:31.294529  5141 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:31.293120  5265 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.322439 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:31.320214 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:31.321151 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.323299 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.324448 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:31.328626 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:31.330195 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.334946 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:31.337111 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.337760 30955 leader_election.cc:400] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19.
I20260709 12:26:31.338187 30955 leader_election.cc:403] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:26:31.361208 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:31.362076 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:31.363082 31060 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:31.362942 31141 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:31.365907 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.367643  5142 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.369779  5142 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:31.356171  4949 raft_consensus.cc:2764] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:26:31.370829 31059 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 12:26:31.371122 30987 raft_consensus.cc:1240] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:31.371766  5258 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:31.374584 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2709) took 1070 ms. Trace:
I20260709 12:26:31.374967 31142 rpcz_store.cc:276] 0709 12:26:30.304458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.513996 (+209538us) service_pool.cc:224] Handling call
0709 12:26:30.514336 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.304955 (+790619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.374360 (+ 69405us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54001e91d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:31.374526 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816616320}
W20260709 12:26:31.375617 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49794.5us
I20260709 12:26:31.378820 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:31.379144 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:31.303392  5087 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 12:26:31.384565 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.387036 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:31.388690 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 470125us
I20260709 12:26:31.334719  5248 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:31.333364  5060 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.391752 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.336620  4985 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:31.299855  5186 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:31.388280  5340 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:31.333134  5260 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 12:26:31.427224 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:31.386700  5258 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:31.426614  5176 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 12:26:31.457480 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:31.458240 30988 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:31.431603  5328 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.419090  5207 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.432641  5365 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:31.434585  5248 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:31.435437  5348 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:26:31.436058  5355 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 12:26:31.436950  5356 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:26:31.439685  5290 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:31.440651  5093 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.441486  5109 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:31.392666  5205 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:31.451354 30983 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:26:31.491525 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:31.452318 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.492455 31062 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:31.393457  4949 raft_consensus.cc:2764] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:26:31.415460  5113 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.455062 30987 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:31.455585 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:31.460120  5275 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:31.459262 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:31.463675  5197 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.392910  5358 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.431805  5364 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:26:31.454053  5345 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.210s	user 0.001s	sys 0.001s
I20260709 12:26:31.482481 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:31.484475 31060 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:31.484639  5248 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.455153  5144 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.422668  5060 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.502483 31065 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
W20260709 12:26:31.501935 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 74779.8us
I20260709 12:26:31.512513 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.474864  5100 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:31.505882 30986 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:26:31.509464 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.565672 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:31.504034 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 147798us
I20260709 12:26:31.496048  5205 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:31.571082  5205 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:31.574743  5363 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:31.577899  5329 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:31.502744  5290 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:31.553511 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 137645us
I20260709 12:26:31.509167  5248 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:31.567549  5325 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:31.581859  5060 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.594374 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2754) took 1214 ms. Trace:
I20260709 12:26:31.594758 30985 rpcz_store.cc:276] 0709 12:26:30.379788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.859353 (+479565us) service_pool.cc:224] Handling call
0709 12:26:30.859840 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.860134 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:31.248550 (+388416us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.473967 (+225417us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.573623 (+ 99656us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b180045a7e0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:26:31.574006 (+   383us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.574150 (+   144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.574195 (+    45us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:31.585164 (+ 10969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.585231 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.594300 (+  9069us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2189,"prepare.queue_time_us":197283,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":116,"raft.queue_time_us":128362,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":147,"spinlock_wait_cycles":11354752,"thread_start_us":550610,"threads_started":2,"wal-append.queue_time_us":259458}
I20260709 12:26:31.505578  5093 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:26:31.583357 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:31.585187 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:31.585976 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:31.564074 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51632.4us
I20260709 12:26:31.568670 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:31.582583 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 1277) took 1041 ms. Trace:
I20260709 12:26:31.612461 31136 rpcz_store.cc:276] 0709 12:26:30.541434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.570265 (+ 28831us) service_pool.cc:224] Handling call
0709 12:26:31.582430 (+1012165us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54001ea0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:31.582530 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080243456}
I20260709 12:26:31.503054  5113 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.614866  5113 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:31.598378 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2977) took 1030 ms. Trace:
I20260709 12:26:31.615403 31064 rpcz_store.cc:276] 0709 12:26:30.568235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.568396 (+   161us) service_pool.cc:224] Handling call
0709 12:26:30.568742 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.569167 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:30.591042 (+ 21875us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.038767 (+447725us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.042943 (+  4176us) log.cc:844] Serialized 26 byte log entry
0709 12:26:31.043127 (+   184us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:31.043174 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.043258 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.598147 (+554889us) raft_consensus.cc:1685] finished
0709 12:26:31.598167 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.598320 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26202,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":2234,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":17574,"thread_start_us":447192,"threads_started":1,"wal-append.queue_time_us":701207}
I20260709 12:26:31.599427 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:31.617681 30986 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:31.601639  5197 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:31.605625 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:31.619125 31060 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:31.611653 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:31.620460 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.620740  5207 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:31.621438  5207 raft_consensus.cc:2764] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:26:31.555598  5308 consensus_queue.cc:1059] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 12:26:31.622390  4949 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:31.624181 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 55652.5us
I20260709 12:26:31.554102  5144 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.627872 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:31.626417 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:31.629406 31064 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:31.629503 30958 leader_election.cc:400] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Vote denied by peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417) with higher term. Message: Invalid argument: T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22.
I20260709 12:26:31.630709 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:31.637218 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:31.637813 30986 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:31.640594 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.641332 31137 raft_consensus.cc:2410] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:26:31.642081 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.642649 31142 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.642757 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:31.642781 31136 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:26:31.647369 31140 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:31.650758 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:31.653383 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:31.653970 31694 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:31.654731 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:31.655278 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.660887 31060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45809554a739456ea0a596484ede5ac7"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:31.661170 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:31.663439 31065 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:31.664136 31142 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:31.666113 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:31.667482 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:31.629061  5278 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:31.670275 31691 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:31.661492 31060 raft_consensus.cc:606] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:31.671675 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:31.671991  5370 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:26:31.672510 31695 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:31.673143  5364 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:31.673560  5202 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:31.674203  5376 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:26:31.675172  5373 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 12:26:31.675889  5375 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:26:31.676595  5374 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:26:31.677467  5248 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.665812  5109 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:26:31.680900  5144 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.585170  5100 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.662506  5113 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:31.644615  5119 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:26:31.683615 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:31.786707 31137 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:31.653301  5325 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:26:31.644974  5087 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:26:31.686514  5343 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.652s	user 0.008s	sys 0.002s
I20260709 12:26:31.669288 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:31.764279  5351 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:31.694370  5379 raft_consensus.cc:993] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:31.698434 31144 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:31.700731 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:31.771538  5380 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:31.795369 31694 raft_consensus.cc:2410] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:26:31.709998 31661 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:31.783667  5113 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.712242 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:31.717346  5358 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19.
I20260709 12:26:31.721403 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:31.723289 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:31.724306 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.729094 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:31.736865 31063 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:31.801381 31142 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:31.635661  5360 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:31.658914  5152 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.751899 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:31.805631 31696 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:31.752173 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:31.756579 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.800748 30991 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 20: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:31.635483  5258 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.115s
I20260709 12:26:31.815444  5100 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.816026  5258 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:31.816402  5258 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:31.259011 30992 rpcz_store.cc:276] 0709 12:26:28.277387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.267126 (+1989739us) service_pool.cc:224] Handling call
0709 12:26:30.267423 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.267621 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:30.620839 (+353218us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.249265 (+628426us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.249578 (+   313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.249632 (+    54us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:31.249692 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.249773 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.249912 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":672425,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":47187,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":1781,"spinlock_wait_cycles":3463040,"thread_start_us":970056,"threads_started":3,"wal-append.queue_time_us":545760}
I20260709 12:26:31.817250  5258 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.761945 31143 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:31.763757 31065 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:26:31.819296 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 1032) took 1450 ms. Trace:
I20260709 12:26:31.819692 31689 rpcz_store.cc:276] 0709 12:26:30.368506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.369058 (+   552us) service_pool.cc:224] Handling call
0709 12:26:30.369357 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.436388 (+ 67031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.436420 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:30.436891 (+   471us) log.cc:844] Serialized 52 byte log entry
0709 12:26:30.737453 (+300562us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:30.737493 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.737582 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.685951 (+948369us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:31.819041 (+133090us) raft_consensus.cc:1685] finished
0709 12:26:31.819062 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.819237 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1249,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":13939,"spinlock_wait_cycles":17074048,"thread_start_us":300260,"threads_started":1,"wal-append.queue_time_us":558951}
I20260709 12:26:31.820405  4942 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.821511  4942 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.794952  5087 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:31.823007 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2853) took 1459 ms. Trace:
I20260709 12:26:31.823709  5087 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.800427  5113 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:31.764813 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 103759us
I20260709 12:26:31.827378 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:31.827574 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:31.828644 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 99618.5us
I20260709 12:26:31.828927  5144 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:31.829352  5144 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:31.724040  4985 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:31.829845 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.830132  5277 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:31.830317  5206 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:31.830598  5277 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.830518  5325 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:31.831108  5277 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:31.831292  5206 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:31.831664  5206 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:31.800724 30984 raft_consensus.cc:2376] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22. Re-sending same reply.
I20260709 12:26:31.832482  5206 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.833350 30984 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:31.833590  5206 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:31.834360  5379 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:31.834853  5379 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.835481 30984 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:26:31.835772  5110 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:31.746578  5364 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.836845  5379 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.837215  5364 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:31.838259  5248 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.839820  5106 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:31.630661  5353 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.840358  5106 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.857297  5106 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:31.857429  5360 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:31.858949  5277 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.860947  5277 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:31.862314  5202 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:31.863139  5093 raft_consensus.cc:3055] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:31.784211 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:31.863549  5093 raft_consensus.cc:740] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:31.863921  4985 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.692879  5265 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:31.864403  5093 consensus_queue.cc:260] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.865311  5265 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.795657  5351 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:31.867455  5265 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:31.867720  5351 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:31.700814  5300 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:31.868709  5378 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:31.868482  5300 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.801115  5262 raft_consensus.cc:1081] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:31.793984 30986 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.701491 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:31.796216 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 86293.8us
I20260709 12:26:31.871301 31694 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 12:26:31.871655 31694 pending_rounds.cc:85] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 13
I20260709 12:26:31.871886 31694 pending_rounds.cc:107] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 12:26:31.872107 31694 raft_consensus.cc:2889] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.772019  5278 consensus_queue.cc:1059] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
W20260709 12:26:31.810585  5372 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:26:31.796329  5328 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.814934  5152 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.829509  5207 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:31.829775  5144 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.773487 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:31.841300 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:31.906672 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 65445.6us
W20260709 12:26:31.907598 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.908358 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:31.843963 31106 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:31.845875 31696 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:31.865887  5093 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:31.852641 30983 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:31.855230 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:31.655988  5033 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:31.850891 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:31.927361 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 76569us
I20260709 12:26:31.857297  5353 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:31.881603 31061 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:26:31.824811  5281 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 12:26:31.940896 30958 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:31.941780 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:31.946741 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:31.947762 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:31.902457 31062 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:26:31.948071 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:31.904408 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:31.845907 31142 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
W20260709 12:26:31.911055 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 67168us
I20260709 12:26:31.950163 31697 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 22. Current term is 23. Ops: [20.12-22.13]
I20260709 12:26:31.921192 30988 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:31.910145  5358 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.923405 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.955875 31141 raft_consensus.cc:2368] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19.
I20260709 12:26:31.955873 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:31.940382 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:31.898104 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:31.957547 31068 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:31.823434 31135 rpcz_store.cc:276] 0709 12:26:30.363214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.784722 (+421508us) service_pool.cc:224] Handling call
0709 12:26:30.915296 (+130574us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.939543 (+ 24247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.939589 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:30.951199 (+ 11610us) log.cc:844] Serialized 52 byte log entry
0709 12:26:31.303634 (+352435us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:31.303674 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.303740 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.441971 (+138231us) raft_consensus.cc:1685] finished
0709 12:26:31.441992 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.822749 (+380757us) spinlock_profiling.cc:243] Waited 61.3 ms on lock 0x7b5400080050. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:31.822946 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40512,"raft.queue_time_us":24134,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":7084,"spinlock_wait_cycles":141045504,"thread_start_us":310564,"threads_started":1,"wal-append.queue_time_us":472384}
I20260709 12:26:31.951534 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.921739 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:31.927383  5257 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.952297 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:31.952922 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:31.953101 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:31.954861 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.940863  5265 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:31.911903  5258 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:31.971088 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.910687  5278 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:31.915302  5365 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:31.926083 30986 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:31.944113  5380 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:31.967882 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:31.970063 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:31.949542  5376 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:31.906401  5328 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:31.959157  5141 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:31.940655  5248 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:26:31.968560  5265 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.036460 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:32.038158 31142 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:26:31.955412  5205 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 12:26:32.048918 31142 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:26:32.039469 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 71654.9us
I20260709 12:26:31.949287  5281 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.042618 31135 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:32.030979  5380 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:31.972738  5290 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:32.050889 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:32.041059 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:32.031008  5329 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:31.962646  5359 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:26:32.117515  5359 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:32.117991  5359 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:32.118970  5359 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.120406  5359 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:32.043537  5141 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.054368 31142 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:26:31.965662  5358 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.121646  5141 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.122069  5377 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:26:32.123005 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 65115.9us
I20260709 12:26:32.123644  5358 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.081508  5216 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:26:32.125573 31142 pending_rounds.cc:85] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 13
I20260709 12:26:32.125926 31142 pending_rounds.cc:107] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:26:32.126241 31142 raft_consensus.cc:2889] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:32.087379 31139 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:32.087996  4876 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:32.128278  4876 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.057719  5389 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:32.052158  5100 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:32.131409  5389 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:32.131381  5100 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.101189  5328 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.103168  5281 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.134646  4942 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:32.103608  5329 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:32.136796  5366 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:32.136907  5329 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.103967  5402 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:32.051266  5265 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:32.105175  5399 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.971176  5257 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.138666  5399 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.140000  5257 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.046360  5376 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:32.034286  5278 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.140767  5399 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.141117  5376 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.142113  5278 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.130738  4876 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:32.089154  5390 log.cc:927] Time spent T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:26:32.134028  5100 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:32.063802  5380 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:32.148317 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 276380us
I20260709 12:26:32.150673  5205 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:32.151681  5205 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:32.137769  5300 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:32.214397 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.211277 31661 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:32.215108 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 258017us
I20260709 12:26:32.132711  5389 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:32.116633  5354 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.000s
I20260709 12:26:32.218645 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:32.152601  5281 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:26:32.223567 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.223358  5364 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.232019 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.232213 31691 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.237169 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.237979 31696 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:32.208751  5119 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:32.242996 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 315436us
I20260709 12:26:32.244112 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:32.244642  5344 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:32.245438 30958 leader_election.cc:400] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19.
I20260709 12:26:32.231482  5122 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.249943 31695 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:32.251099 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 314727us
I20260709 12:26:32.208585  5216 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:26:32.227882  5281 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:32.230150  4985 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:32.240746  5197 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:32.254554 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.267206 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:32.255168 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.256301 31696 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:32.260947 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.260522 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:32.221365  5176 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:26:32.245090  5399 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:32.271355 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.269631 31694 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:32.228678  5353 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:26:32.284448  5114 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.287075  5100 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.288806 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:32.294495 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 243655us
I20260709 12:26:32.298890 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:32.309343 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:32.310307 31064 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:32.299536 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:32.310513 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.311450 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.312942 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.313194 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.313791 31693 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:32.239959  5113 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.328428 31694 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:32.330247 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:32.267372  5087 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:32.230089  5378 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:26:32.331817 31062 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:32.334800 31698 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:32.287949  5300 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.332994 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:32.251528  5354 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:26:32.349740 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:32.292865  5328 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:32.248129  5119 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:32.352593 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 108546us
I20260709 12:26:32.358850 31137 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:32.360357 30991 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:32.306959  4876 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:32.297602  5399 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.399092 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:32.354317  5300 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.416461 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:32.329315  5100 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.320021  5114 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:32.421798  5398 log.cc:927] Time spent T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.159s	user 0.002s	sys 0.007s
I20260709 12:26:32.398897  5396 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:32.360553  5277 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.353976  4985 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:26:32.429196 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:32.486758 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 139193us
I20260709 12:26:32.393949  4876 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.496315 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:32.497553 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:32.498926 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 243137us
I20260709 12:26:32.500782 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:32.537896  5329 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:32.499015  5351 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:32.547509  5414 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.544903  5113 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:32.548417  5408 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:32.548214  5414 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.548887  5408 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:32.549280  5408 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.466511  5277 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.461282  5265 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:32.554301  5394 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:26:32.554417  5414 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.562875  5426 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:32.563407  5426 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:32.564236  5426 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.565377  5380 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.503615  5087 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:26:32.502151  5396 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:26:32.536265  5278 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.540949  5364 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.566994  5278 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.569202  5278 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.560689  4949 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:32.569847  4949 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:32.570806  4949 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:32.573248  5395 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:26:32.583165  5421 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.583700  5421 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.585740  5421 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.587996 31135 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:32.621284 31139 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 23. Current term is 24. Ops: [23.13-23.13]
W20260709 12:26:32.578953  5404 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:26:32.625484 31058 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.647483  5152 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.652390 31058 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:32.615593  5421 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:32.667212 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 166504us
I20260709 12:26:32.668339 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.670787 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.677430 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.658438  5119 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.662809  5106 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.666033  5421 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:32.683573  5407 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:26:32.683713 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.709497 30989 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:26:32.709901 30989 pending_rounds.cc:85] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 10
I20260709 12:26:32.710120 30989 pending_rounds.cc:107] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:26:32.710384 30989 raft_consensus.cc:2889] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:32.716476  5093 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.000s
I20260709 12:26:32.696063  5114 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.702592  5399 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.703187  4876 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:32.671159  5406 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.706521  5100 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:32.707921 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:32.744292 31142 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:32.695591  5277 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.718231  5401 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.718699 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:32.720188 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 3029) took 1083 ms. Trace:
W20260709 12:26:32.720414  5206 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.839s	user 0.004s	sys 0.004s
W20260709 12:26:32.721797  5141 consensus_meta.cc:287] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.003s
I20260709 12:26:32.722950 31062 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:32.723408  4673 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:32.727427 31061 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:26:32.727825 31059 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:32.727614  5380 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.728168  5421 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:32.730795 30876 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2439e7934b98446ca4e7d82381e8a499 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e9cccf7eb2a545ef8c2169b67ff261bf is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:32.695597  5265 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.729890 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:32.734151 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 236686us
I20260709 12:26:32.735944 31694 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:32.703905  5106 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.704218  5300 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.683537  5119 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:26:32.745225 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:32.716866  5144 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.781s	user 0.009s	sys 0.000s
I20260709 12:26:32.747258 30985 rpcz_store.cc:276] 0709 12:26:31.637071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.642849 (+  5778us) service_pool.cc:224] Handling call
0709 12:26:31.643113 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.907340 (+264227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.907388 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.915966 (+  8578us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.250823 (+334857us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:32.250879 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.719788 (+468909us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:32.719838 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.719897 (+    59us) raft_consensus.cc:1685] finished
0709 12:26:32.719910 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.720129 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83181,"raft.queue_time_us":72411,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":25709,"spinlock_wait_cycles":419504896,"thread_start_us":287396,"threads_started":1,"wal-append.queue_time_us":612843}
I20260709 12:26:32.757284  5100 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:32.744892  5406 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.782998 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:32.785422 30876 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54697.7us
I20260709 12:26:32.785190 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:32.785928 31694 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:32.821408 30991 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:32.783528 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:32.791735  5393 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.837s	user 0.002s	sys 0.001s
I20260709 12:26:32.802750 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:32.821458 31140 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:32.808656 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.807768 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:32.782898  5258 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.751s	user 0.016s	sys 0.004s
I20260709 12:26:32.795972 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:32.817938 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.822238 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.825126 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:32.825958 31142 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 23: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:32.823102 31141 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:32.806803 31144 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
W20260709 12:26:32.806455  5437 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:26:32.828835 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:32.832576 31140 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:32.832726  5100 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:32.833089  5438 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:32.822019 31694 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.820674  5380 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:32.834358 31694 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.834429  5380 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.830761 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:32.836689  5380 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.837087 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:32.838413  5206 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:32.839648 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:32.841094  5380 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:32.841784  5380 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:32.842756  5359 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.722s	user 0.006s	sys 0.006s
I20260709 12:26:32.844026 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1275) took 1088 ms. Trace:
I20260709 12:26:32.844722  5406 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.817423  5197 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:26:32.842710  5380 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.845505  5197 consensus_queue.cc:1243] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:32.801389  5106 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:32.755158  5141 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.846349 30988 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 15: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:32.847244 31691 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:32.827545  5434 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.847867  5141 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.829772 30990 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:32.820931  5144 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.844406 30984 rpcz_store.cc:276] 0709 12:26:31.756108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.836167 (+ 80059us) service_pool.cc:224] Handling call
0709 12:26:31.836433 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.840999 (+  4566us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:31.863048 (+ 22049us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.947582 (+ 84534us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.947853 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.040165 (+ 92312us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.055944 (+ 15779us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.056139 (+   195us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:32.056184 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.056274 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.144724 (+ 88450us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b18003d3540. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:32.572195 (+427471us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b4800080058. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef07bdaa1 00007f2ef07b4b37 00007f2ef07b49ca 00007f2ef08ac26f 00007f2ef08abf37 00007f2ef08c055c 00007f2ef08c0508 00007f2ef08c0488 00007f2ef08c0443 00007f2ef08bf6ec 00007f2eef7e9a96
0709 12:26:32.747196 (+175001us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b18003d3540. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a7522 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:32.791220 (+ 44024us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:32.843825 (+ 52605us) raft_consensus.cc:1685] finished
0709 12:26:32.843849 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.843967 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54637,"prepare.queue_time_us":831916,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":99,"raft.queue_time_us":102270,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"spinlock_wait_cycles":422166912,"thread_start_us":121055,"threads_started":2,"wal-append.queue_time_us":63782}
I20260709 12:26:32.821100  5277 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.757305  4673 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.852449 31694 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 12:26:32.862135 31694 pending_rounds.cc:85] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 14
I20260709 12:26:32.862464 31694 pending_rounds.cc:107] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:26:32.862799 31694 raft_consensus.cc:2889] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:32.858625 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:32.853025  5119 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.859735  5399 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:32.883927 31068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60b6187f32774daba89763099dabeb2a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:60354
W20260709 12:26:32.864998  5379 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.012s	sys 0.009s
I20260709 12:26:32.751914  5401 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.789188  5420 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.098s
W20260709 12:26:32.882145  4991 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:26:32.806209  5408 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.862534  5412 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.888567 31068 raft_consensus.cc:606] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Received request to transfer leadership to TS a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:32.883278  5325 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.021s	sys 0.029s
I20260709 12:26:32.886812  4876 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:26:32.887027  5426 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:26:32.898415  5445 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:32.889922  5093 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.901801 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:32.907790 31695 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 1388) took 1169 ms. Trace:
W20260709 12:26:32.890074  5379 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.012s	sys 0.010s
I20260709 12:26:32.865141  5277 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.922909 31695 rpcz_store.cc:276] 0709 12:26:31.750562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.750712 (+   150us) service_pool.cc:224] Handling call
0709 12:26:31.773581 (+ 22869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.257606 (+484025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.907520 (+649914us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54005e7754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:32.907719 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1095087104}
W20260709 12:26:32.914143  5325 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.021s	sys 0.030s
I20260709 12:26:32.931027 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:32.848100  5434 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.931509 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:32.936156 31139 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:32.891634  5119 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.922508  5106 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:32.951812  5030 raft_consensus.cc:3055] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:32.951588  5106 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.952225  5030 raft_consensus.cc:740] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:32.953214  5030 consensus_queue.cc:260] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.955119  5030 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:32.955933  5207 raft_consensus.cc:993] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:32.956456  5207 raft_consensus.cc:1081] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:32.957095  5421 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.941237 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.958480  5119 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.941426 31139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45364
I20260709 12:26:32.959297 31139 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:32.924727  5418 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 12:26:32.943449  5445 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.919128  5412 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.925657  5422 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:32.944929  5408 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.946360  4673 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.946595  5277 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.948355  5434 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.938555 31068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:32.954124 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2762) took 1259 ms. Trace:
I20260709 12:26:32.992632 31068 raft_consensus.cc:606] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:32.992579 30983 rpcz_store.cc:276] 0709 12:26:31.710647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.828204 (+117557us) service_pool.cc:224] Handling call
0709 12:26:31.852223 (+ 24019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.912669 (+ 60446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.953937 (+1041268us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54003453d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:32.954064 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54566656}
I20260709 12:26:32.958648 30986 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:32.928791  5093 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:32.959666 31139 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.959918 30991 raft_consensus.cc:2393] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:32.965704 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:32.985244 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:32.997315 31138 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:32.998517 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:32.987088 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:32.960012  5106 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.011752 31138 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:33.014585 31139 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:32.987645  5422 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:33.028712 31068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e39858333670419fb20dc8adde042dd2"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:33.029302 31068 raft_consensus.cc:606] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:33.029641 31068 raft_consensus.cc:614] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:33.033939  5144 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:32.980116  5412 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:32.976044  5445 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.036607  5445 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:33.037184  5401 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.041643 31139 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.038146 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:33.043155 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:33.025187 31138 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:33.026087  5380 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:33.026347  5421 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.026688  5438 raft_consensus.cc:993] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:26:33.044883 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:33.046734 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:33.055908 30990 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:33.056326 30990 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:33.057265 30990 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.053427 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:33.039206  5093 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:33.059618 30990 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:33.065255 31139 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:33.066762 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:33.070147 31139 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:33.072890  5144 raft_consensus.cc:1081] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Signalling peer a517aadb0ae04b89a13779d7898dfa0c to start an election
I20260709 12:26:32.978302  5281 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.045835  5380 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.095606 31696 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60b6187f32774daba89763099dabeb2a"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
 from {username='slave'} at 127.0.0.1:51374
I20260709 12:26:33.096344 31696 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:33.096791 31696 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:33.097498 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.102397 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:33.103037 30987 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:33.104598 31698 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1385) took 1435 ms (client timeout 1859 ms). Trace:
W20260709 12:26:33.113853 31698 rpcz_store.cc:269] 0709 12:26:31.669437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.669557 (+   120us) service_pool.cc:224] Handling call
0709 12:26:33.104423 (+1434866us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54004443d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:33.104526 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147878144}
I20260709 12:26:33.096567  5207 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:33.124514  5207 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:33.137826  5205 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:33.138286  5205 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:33.139182  5205 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:33.141141 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 797388us
I20260709 12:26:33.143471 31068 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 12:26:33.143813 31068 pending_rounds.cc:85] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 10
I20260709 12:26:33.143981 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 3001) took 1223 ms. Trace:
I20260709 12:26:33.144173 30986 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 12:26:33.144644 30986 pending_rounds.cc:85] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 11
I20260709 12:26:33.144469 31058 rpcz_store.cc:276] 0709 12:26:31.920478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.921522 (+  1044us) service_pool.cc:224] Handling call
0709 12:26:33.143818 (+1222296us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54002ce7d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:33.143926 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533616000}
I20260709 12:26:33.145134 30986 pending_rounds.cc:107] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:26:33.145455 30986 raft_consensus.cc:2889] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:33.146488 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.144039 31068 pending_rounds.cc:107] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 12:26:33.149003 31068 raft_consensus.cc:2889] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:33.154326 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.155301 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.155676 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.158687 31696 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.160303 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.163359 31694 raft_consensus.cc:1240] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:33.167104 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.169502 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:33.215780 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.164774 31058 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:33.216526 31694 raft_consensus.cc:2393] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:33.217474 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.219162 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.240427 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.224577 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:33.168845 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.223009 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.191229 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:33.096845  5144 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:33.239985 31067 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:33.252074 31691 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:33.241329 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.243099 31696 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:26:33.098189  5380 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:33.253847 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:33.270925 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.271955 31064 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:33.196278 30990 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:33.257517 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:33.273067 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:33.273763 30990 raft_consensus.cc:2368] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17.
I20260709 12:26:33.274686 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:33.175146  5113 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:33.277282 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:33.277639 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.221369 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:33.278209 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:33.278669 30986 raft_consensus.cc:2368] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19.
I20260709 12:26:33.278788 31061 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.259238 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.252125 31065 raft_consensus.cc:2410] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:26:33.279130 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:33.139256  5391 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.086s	user 0.001s	sys 0.002s
I20260709 12:26:33.197043  4876 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.125768  5207 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.175123  5400 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.281845 31062 raft_consensus.cc:2410] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:26:33.175554  5281 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:33.276767 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:33.288931 30958 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.540s (SENT) [suppressed 15 similar messages]
I20260709 12:26:33.266863  5412 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.291667 30987 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:26:33.292136 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:33.266538  5421 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:33.294109 30986 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:33.270557  5186 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.290050  5434 consensus_queue.cc:1059] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:26:33.296387  5186 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.297073  5412 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.298667  5186 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:33.299234  5412 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.284034  5113 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.302380 31067 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:33.306257 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:33.254786  5448 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:26:33.281944  5430 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:33.311733 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:33.096843  5422 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.269261  5144 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.322340 31061 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
W20260709 12:26:33.310134  5389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.006s	sys 0.009s
I20260709 12:26:33.313371  5430 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.289793  4876 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.290400  5152 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:26:33.290839  5400 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.339226 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2867) took 1003 ms. Trace:
W20260709 12:26:33.327576  5389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.007s	sys 0.009s
I20260709 12:26:33.310495  5421 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.335007  5430 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:33.324785  5325 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:33.348215 31136 rpcz_store.cc:276] 0709 12:26:32.335555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.937566 (+602011us) service_pool.cc:224] Handling call
0709 12:26:32.938989 (+  1423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.939291 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.939319 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.959125 (+ 19806us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.046010 (+ 86885us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:33.046052 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.046130 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.338952 (+292822us) raft_consensus.cc:1685] finished
0709 12:26:33.338981 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.339152 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":408,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":86533,"threads_started":1,"wal-append.queue_time_us":280849}
I20260709 12:26:33.335117  5422 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:33.353878 31141 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:33.358031 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:33.359117  5113 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.367415  5400 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.354944  5325 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.370839  5421 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.376621  5325 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.379496  5095 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.380035  5095 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:33.380492  5413 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.835s	user 0.001s	sys 0.001s
W20260709 12:26:33.381412 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.382361  5095 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:33.386658  5455 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.331s	user 0.002s	sys 0.000s
I20260709 12:26:33.387652  5033 raft_consensus.cc:3055] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:33.388093  5033 raft_consensus.cc:740] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:33.388914  5033 consensus_queue.cc:260] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:33.404560  5365 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.357s	user 0.005s	sys 0.000s
W20260709 12:26:33.405323  5365 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.361s	user 0.005s	sys 0.001s
I20260709 12:26:33.405576  5365 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:33.408922  5461 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:26:33.408845  5380 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:33.444837 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 155692us
I20260709 12:26:33.449417 30986 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:26:33.451807 30987 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:33.456657 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 2999) took 1602 ms. Trace:
I20260709 12:26:33.458984 31058 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:33.457159 31060 rpcz_store.cc:276] 0709 12:26:31.854113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.854291 (+   178us) service_pool.cc:224] Handling call
0709 12:26:31.854618 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.875516 (+ 20898us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:31.908265 (+ 32749us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.246046 (+337781us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.252137 (+  6091us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.252288 (+   151us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:32.254025 (+  1737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.254093 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.379971 (+1125878us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:33.455945 (+ 75974us) raft_consensus.cc:1685] finished
0709 12:26:33.456109 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.456401 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34248,"prepare.queue_time_us":35174,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":20868,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":20709,"spinlock_wait_cycles":6671232,"thread_start_us":298210,"threads_started":1,"wal-append.queue_time_us":546920}
I20260709 12:26:33.462387 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:33.463449 30990 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:33.469308 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:33.472041 31033 leader_election.cc:400] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17.
I20260709 12:26:33.411079  5430 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.634083 31033 leader_election.cc:403] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:33.390825  5033 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.639089 30990 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:33.641423  5430 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.641541 31694 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:33.642206  5467 raft_consensus.cc:993] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:33.644923  5448 raft_consensus.cc:1081] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:33.638962  5472 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:26:33.652046  5380 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.651650 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:33.654309 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 182821us
I20260709 12:26:33.655298  5467 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 16. Current term is 17.
I20260709 12:26:33.655432 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:33.656803  5448 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:33.642505  5422 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:33.658771  5422 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.661847  5422 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.657976 31033 leader_election.cc:400] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19.
W20260709 12:26:33.644966  5470 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.231s	user 0.006s	sys 0.012s
W20260709 12:26:33.671227 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 202113us
I20260709 12:26:33.673496 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:33.673820  5095 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:33.674358 31136 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:33.675202 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.674629  5095 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.677865  5095 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:33.686321  5144 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:33.686921 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.678036  5422 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.687145 30985 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:33.679009 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.687785  5422 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.688431 31696 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:33.688715 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2752) took 3309 ms. Trace:
I20260709 12:26:33.688968 31697 raft_consensus.cc:1240] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:26:33.689468  5095 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:33.690716  5422 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.690287  5095 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.689118 30989 rpcz_store.cc:276] 0709 12:26:30.379337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.855746 (+476409us) service_pool.cc:224] Handling call
0709 12:26:30.855960 (+   214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.710807 (+1854847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.688162 (+977355us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b540014dcd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:33.688507 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4261958016}
I20260709 12:26:33.536372  5365 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.681421 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.694599 31061 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:33.694784 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.681658 31060 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:33.682511 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:33.697089 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.682636 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.683317  5472 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:33.682951  5430 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:33.685563 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 3039) took 1017 ms. Trace:
I20260709 12:26:33.680222 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.696733 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.696784 31065 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:33.670202  5325 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:33.698856 30988 rpcz_store.cc:276] 0709 12:26:32.668456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.678655 (+ 10199us) service_pool.cc:224] Handling call
0709 12:26:33.685080 (+1006425us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54003458d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:33.685329 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315057536}
I20260709 12:26:33.703819 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:33.705183  5399 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:33.705487 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.706506 31697 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:33.699811  5430 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.705732  5399 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:33.707738  5472 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:33.709869  5460 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.374s	user 0.000s	sys 0.001s
I20260709 12:26:33.706148  5325 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:33.712342  5463 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.375s	user 0.001s	sys 0.002s
I20260709 12:26:33.714448  5110 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.715646 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1299) took 1062 ms. Trace:
I20260709 12:26:33.715378  5110 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.716066 31138 rpcz_store.cc:276] 0709 12:26:32.653488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.044885 (+391397us) service_pool.cc:224] Handling call
0709 12:26:33.045126 (+   241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.060125 (+ 14999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.060172 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.083538 (+ 23366us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.282504 (+198966us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:33.282540 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.282608 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.711400 (+428792us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:33.715090 (+  3690us) raft_consensus.cc:1685] finished
0709 12:26:33.715118 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.715429 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14653,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":21689,"spinlock_wait_cycles":12412160,"thread_start_us":192054,"threads_started":1,"wal-append.queue_time_us":242851}
I20260709 12:26:33.719727 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:33.687144 31139 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:33.693205  5095 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.726524 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:33.727125  5205 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.727444 31696 raft_consensus.cc:2376] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16. Re-sending same reply.
I20260709 12:26:33.707321  5430 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.729317 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:33.730127 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.730784 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.731230 31695 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:33.713323  5325 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.738705 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.720479 31138 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
W20260709 12:26:33.724112 31137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2727) took 1797 ms (client timeout 2051 ms). Trace:
W20260709 12:26:33.742587 31137 rpcz_store.cc:269] 0709 12:26:31.924937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.925037 (+   100us) service_pool.cc:224] Handling call
0709 12:26:33.721787 (+1796750us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b540006e0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:33.722015 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":942731648}
I20260709 12:26:33.744480 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:33.745447 31137 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.725128 31693 raft_consensus.cc:1240] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:33.726336 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.729295  5110 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.748272  5472 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.766804 30989 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:26:33.777318 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:33.780092 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:33.782337 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.789708  5401 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:33.790860  5401 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:33.792629 31143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2859) took 2138 ms (client timeout 2112 ms). Trace:
W20260709 12:26:33.793215 31143 rpcz_store.cc:269] 0709 12:26:31.654165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.654524 (+   359us) service_pool.cc:224] Handling call
0709 12:26:33.792115 (+2137591us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54000947d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:33.792354 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244628864}
I20260709 12:26:33.802386 31136 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:33.804674 31139 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:33.811975 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.812371 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:33.812849 31695 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:33.815210 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2738) took 1071 ms. Trace:
W20260709 12:26:33.816625 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 158958us
I20260709 12:26:33.815528 31139 rpcz_store.cc:276] 0709 12:26:32.734778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.763164 (+1028386us) service_pool.cc:224] Handling call
0709 12:26:33.786606 (+ 23442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.805044 (+ 18438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.805872 (+   828us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:33.818086 31138 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:33.805980 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:33.813907  5368 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:26:33.826310  5281 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.010s
I20260709 12:26:33.823263  5368 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:33.828878 31691 consensus_meta.cc:287] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.015s
I20260709 12:26:33.848239 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:33.849059 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 83070.5us
I20260709 12:26:33.851203  5095 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:26:33.855466 31144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2725) took 1931 ms (client timeout 1540 ms). Trace:
W20260709 12:26:33.855901 31144 rpcz_store.cc:269] 0709 12:26:31.924352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.926646 (+  2294us) service_pool.cc:224] Handling call
0709 12:26:33.855093 (+1928447us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54001eafd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:33.855260 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1803204352}
I20260709 12:26:33.847259 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:33.858945 31691 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:33.845690  5368 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:33.877436 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:33.846884  5401 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:33.879658 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 60607.6us
I20260709 12:26:33.880208 31694 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:33.891295 31696 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:33.888613 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:33.848546 31139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45809554a739456ea0a596484ede5ac7"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45360
I20260709 12:26:33.909636 31139 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:33.910151 31139 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.911224  5365 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:33.887490  5281 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:33.921838 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 78778us
I20260709 12:26:33.923897 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:33.938278  5152 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:33.938748  5152 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:33.939163  5152 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:33.941299  5109 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:33.941730  5109 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.942082  5109 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:33.950573  5258 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:33.887692  4876 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:33.951881 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:33.920253 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:33.955312  5365 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.950589 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1298) took 1297 ms. Trace:
W20260709 12:26:33.959234 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 147120us
I20260709 12:26:33.958030  5365 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.962829 31137 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:33.970202  5030 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.970938  5030 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:33.971367 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:33.972112 31691 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:33.919523  5472 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:33.973373  5030 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:33.973124  5472 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.974046  5325 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.974668  5325 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:33.975394  5472 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.975891 30958 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:33.976780  5325 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:33.976961  5110 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:33.963482  5467 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:33.978633  5467 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:33.979092  5467 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:33.973960  5030 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:34.010504  5060 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.011387  5060 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.013245  5152 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.015530 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:34.015949 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:34.016805 31067 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:34.017166 31067 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:34.017292  5093 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:34.019131 31060 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:34.022914  5106 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:33.989161  5110 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:34.024760 31690 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:34.024932  5110 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.031533 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1284) took 1359 ms. Trace:
I20260709 12:26:34.031939 30985 rpcz_store.cc:276] 0709 12:26:32.671779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.828400 (+156621us) service_pool.cc:224] Handling call
0709 12:26:33.686395 (+857995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.687684 (+  1289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.031329 (+343645us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54003458d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:34.031462 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1972420864}
I20260709 12:26:34.033648 31143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2871) took 1177 ms. Trace:
I20260709 12:26:34.034020 31143 rpcz_store.cc:276] 0709 12:26:32.856755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.794307 (+937552us) service_pool.cc:224] Handling call
0709 12:26:33.825203 (+ 30896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.825675 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:33.845151 (+ 19476us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.033225 (+188074us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.033282 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:34.033379 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.033464 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.033578 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":143446,"prepare.queue_time_us":3080,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"thread_start_us":63252,"threads_started":2}
I20260709 12:26:34.035688  4876 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.036062 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1292) took 2095 ms. Trace:
I20260709 12:26:34.036633 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:34.036495 31135 rpcz_store.cc:276] 0709 12:26:31.940370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.069175 (+128805us) service_pool.cc:224] Handling call
0709 12:26:32.069531 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.588420 (+518889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.035558 (+1447138us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54003679d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:34.035855 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191916928}
I20260709 12:26:34.037384 31692 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:34.037926 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 71993.7us
I20260709 12:26:34.039223 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.040864 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.042526  5445 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:34.043251 31064 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:34.045336  5389 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:34.046629 31692 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:34.046677  5030 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.048539  5186 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:34.049116  5030 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:34.049459  5300 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:34.049858  5300 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:26:34.050884 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 127163us
I20260709 12:26:34.051026  5300 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.054190  5328 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:34.054432  5472 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:34.054623 31067 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.056021  5144 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:34.055670 31067 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:34.036312  4876 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.062781 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:34.062973  4876 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:34.063829 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.064291 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:33.964088  5281 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.068167  5281 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:34.070698  5430 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.071671 31106 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.072808  5430 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:34.073316 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:34.074043 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 85424.7us
W20260709 12:26:34.075454 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 80906.6us
I20260709 12:26:34.084219 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.084968 31058 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
W20260709 12:26:34.085132 30884 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 316) took 50137 ms (client timeout 15000 ms). Trace:
I20260709 12:26:34.085971 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:34.085587 30884 rpcz_store.cc:269] 0709 12:25:43.947927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.948175 (+   248us) service_pool.cc:224] Handling call
0709 12:25:44.281450 (+333275us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5400002210. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee876fb71 00007f2ee87700b2 00007f2ef16e9759 00007f2ef1681a76 00007f2ef1652d58 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc
0709 12:25:44.607414 (+325964us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fe090 00007f2eeb89cf3a 00007f2ef1652e9c 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:34.084868 (+49477454us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:27.458343 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:27.477641 (+ 19298us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:27.477690 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:26:27.477708 (+    18us) tablet.cc:662] Decoding operations
0709 12:26:27.478539 (+   831us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:27.478644 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:27.478671 (+    27us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:26:27.479273 (+   602us) tablet.cc:701] Row locks acquired
0709 12:26:27.479321 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:26:27.479478 (+   157us) write_op.cc:270] Start()
0709 12:26:27.479625 (+   147us) write_op.cc:276] Timestamp: P: 1783599987479415 usec, L: 0
0709 12:26:27.479647 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:26:27.480331 (+   684us) log.cc:844] Serialized 2434 byte log entry
0709 12:26:28.915445 (+1435114us) op_driver.cc:464] REPLICATION: finished
0709 12:26:29.059576 (+144131us) write_op.cc:301] APPLY: starting
0709 12:26:29.059678 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:29.065827 (+  6149us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:29.065929 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:29.129612 (+ 63683us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:29.129835 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 12:26:29.129882 (+    47us) write_op.cc:312] APPLY: finished
0709 12:26:29.142266 (+ 12384us) log.cc:844] Serialized 65 byte log entry
0709 12:26:29.164976 (+ 22710us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:29.180538 (+ 15562us) write_op.cc:454] Released schema lock
0709 12:26:29.185198 (+  4660us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":617659,"reported_tablets":48,"spinlock_wait_cycles":190548480,"child_traces":[["op",{"apply.queue_time_us":143806,"apply.run_cpu_time_us":9250,"apply.run_wall_time_us":160088,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29219,"num_ops":6,"prepare.queue_time_us":123481,"prepare.run_cpu_time_us":32363,"prepare.run_wall_time_us":279134,"raft.queue_time_us":759298,"raft.run_cpu_time_us":9507,"raft.run_wall_time_us":224742,"replication_time_us":1390454,"thread_start_us":650703,"threads_started":4,"wal-append.queue_time_us":270471,"wal-append.run_cpu_time_us":20642,"wal-append.run_wall_time_us":2478697}]]}
I20260709 12:26:34.097009  5276 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:34.097461  5276 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.097790  5276 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:34.098328  5399 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.100980  5399 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:34.101768  5427 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:34.091164 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 75855.8us
I20260709 12:26:33.958745 31140 rpcz_store.cc:276] 0709 12:26:32.653220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.034016 (+380796us) service_pool.cc:224] Handling call
0709 12:26:33.034390 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:33.034829 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:33.035122 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.267688 (+232566us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.271309 (+  3621us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:33.272174 (+   865us) log.cc:844] Serialized 52 byte log entry
0709 12:26:33.272372 (+   198us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:33.272623 (+   251us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.706950 (+434327us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b180045eb00. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a7522 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:33.708906 (+  1956us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:33.949570 (+240664us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b180045eb00. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:26:33.949963 (+   393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.950072 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.950114 (+    42us) raft_consensus.cc:1685] finished
0709 12:26:33.950128 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.950361 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28453,"prepare.queue_time_us":440984,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":2693,"raft.queue_time_us":710837,"raft.run_cpu_time_us":502,"raft.run_wall_time_us":15553,"spinlock_wait_cycles":28967808,"thread_start_us":213126,"threads_started":1,"wal-append.queue_time_us":286847}
I20260709 12:26:34.089871 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:34.056056  5328 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:34.113653 31068 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:26:34.110491 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:34.111461 31139 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.014065  5060 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:34.113582  5328 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:34.117695  5480 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.119598  5480 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.119831  5328 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.120863  5482 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:34.121266  5328 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:34.121412  5482 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:34.121910  5480 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:34.122504 31142 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:34.122579  5482 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.122502  5378 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:26:34.127511 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.127733 30883 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 11 to 14, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 14 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:34.129376  5378 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:34.107945  5445 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:34.130045  5480 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:34.132480  5487 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:34.133102  5487 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.133309  5427 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.135246  5487 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:34.162853 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.154337 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.141381 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:34.157631 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:34.136253 30955 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:34.138463  5401 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
W20260709 12:26:34.192761 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 86736.5us
W20260709 12:26:34.164362 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 91147.8us
I20260709 12:26:34.192409  5378 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:34.179852 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1744) took 1458 ms. Trace:
W20260709 12:26:34.161369 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 81665.1us
W20260709 12:26:34.233929 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.233146 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.136027  5427 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:34.194147  5401 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:34.243856 31106 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.245430 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:34.247967 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:34.248519 31692 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.232170  5378 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.256088 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.152851  5368 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.153498  5152 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:34.196498 30883 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 6 to 10, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 10 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.135339  5276 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:34.206943 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 732382us
I20260709 12:26:34.277822  5276 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:34.279601 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.210528 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:34.108305  4876 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:34.285821 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.160861 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.287031 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:34.211619  5389 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:34.282996 31061 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:34.287866 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 157658us
I20260709 12:26:34.290392  5480 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:34.213218 31139 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:34.291332 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45360 (request call id 2874) took 1328 ms. Trace:
I20260709 12:26:34.291183  5095 consensus_queue.cc:1059] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:34.291711 31139 rpcz_store.cc:276] 0709 12:26:32.963071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.820137 (+857066us) service_pool.cc:224] Handling call
0709 12:26:34.291251 (+471114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:34.212852  5467 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.222548 31067 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:34.161731  5144 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:34.294116  5144 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:34.234521 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.180394  5109 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.253611  5106 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:34.231070 31063 rpcz_store.cc:276] 0709 12:26:32.721713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.722115 (+   402us) service_pool.cc:224] Handling call
0709 12:26:32.722466 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.770440 (+1047974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.772822 (+  2382us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.804448 (+ 31626us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.889067 (+ 84619us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:33.889112 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.939952 (+ 50840us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54003976d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:33.940003 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.179443 (+239440us) raft_consensus.cc:1685] finished
0709 12:26:34.179461 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.179699 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14215,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":9932,"spinlock_wait_cycles":1839266048,"thread_start_us":29396,"threads_started":1,"wal-append.queue_time_us":355042}
I20260709 12:26:34.274964 30883 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.256475  5378 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:34.280985 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.282778  5276 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:34.282974 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 88769us
I20260709 12:26:34.284188 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.284906 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.292122  5095 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:34.287958 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.255304  5399 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:34.292018 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:34.267479  5401 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:34.294535  5144 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:34.271083  5472 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.267307  5445 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:34.306098 31060 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:34.300201 30984 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:34.309202 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2873) took 1367 ms. Trace:
I20260709 12:26:34.312341 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2877) took 1266 ms. Trace:
I20260709 12:26:34.319032 31138 rpcz_store.cc:276] 0709 12:26:33.040903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.822405 (+781502us) service_pool.cc:224] Handling call
0709 12:26:33.888689 (+ 66284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.969857 (+ 81168us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:33.970256 (+   399us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.970757 (+   501us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.306650 (+335893us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.306721 (+    71us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:34.306785 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.306866 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.307002 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2042,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":106769,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":184,"thread_start_us":335604,"threads_started":1}
I20260709 12:26:34.297894  5106 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.321673  5106 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:34.332362 30955 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:34.333312  5430 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:34.333094  5487 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:34.335541  4876 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:34.335988  4876 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:34.336367  4876 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:34.309194  5467 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:34.339737 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 106807us
I20260709 12:26:34.271246  5281 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:26:34.318900 31136 rpcz_store.cc:276] 0709 12:26:32.935242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.803151 (+867909us) service_pool.cc:224] Handling call
0709 12:26:33.803589 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.803953 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:33.817659 (+ 13706us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.269921 (+452262us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.300906 (+ 30985us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:34.300969 (+    63us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:34.301023 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.302287 (+  1264us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.302428 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13058,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":27524,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":37744,"spinlock_wait_cycles":11225472,"thread_start_us":374980,"threads_started":1}
I20260709 12:26:34.340725 31692 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:34.300204  5480 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:26:34.353075 30958 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:34.357502 30985 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:34.371021 30955 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:34.372743 30991 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:26:34.373883 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.375205 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.377084 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.380352  5300 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:34.382362 30990 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:34.385982 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.386929 31064 raft_consensus.cc:2410] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:26:34.388130 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.389313 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:34.389909 31064 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:34.390607 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.391361 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:34.391942 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.390734  5431 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:34.354405  5380 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:26:34.394301 31139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45360
I20260709 12:26:34.392725 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:34.393152 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.397034 31141 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:34.392159 31689 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:34.396138 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.394622 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.411754 31063 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:34.392694 31068 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:34.395788 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.394722 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:34.390189 31064 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:34.394045 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.393500 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.417575 31067 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:34.408002 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.420027 30988 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:34.421196 31135 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:34.411648  5427 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.420281 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.427773 31690 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:34.428495  5033 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:34.427809 31693 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:34.402594  5379 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.430610 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.430985  5379 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.432144 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.432427  5060 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.432962  5060 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.433482 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.356053  5399 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.433285  5379 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:34.441136 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:34.441726 31696 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:34.442077 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.442397 31143 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:34.445547 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.418984  5431 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:34.445770 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:34.448323 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:34.449292  5060 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:34.450160 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:34.450762 31136 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:34.446864 31142 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:34.454085 31137 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:34.471870 30984 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:34.455744 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:34.454844 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.548818 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:34.554101 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.554517 31140 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:34.550537 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 94871.6us
W20260709 12:26:34.557693 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 216710us
I20260709 12:26:34.559484 31694 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:34.559474 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 268607us
W20260709 12:26:34.563598 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.399572  5401 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.570459 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:34.571775 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 218430us
I20260709 12:26:34.573774 30991 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:34.564780 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:34.443656  5427 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.571055 31143 raft_consensus.cc:2376] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16. Re-sending same reply.
I20260709 12:26:34.549820  5276 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:34.555714  5060 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.578070 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:34.581692 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 166501us
I20260709 12:26:34.559109  5487 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:34.592217 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.602766 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.575436  5325 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.602821 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:34.607080 31139 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:34.607524 31139 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:34.589880  5487 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.598300  5482 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.618906 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:34.619009 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:34.622653 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:34.623625  5491 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:34.624478  5418 raft_consensus.cc:3055] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:34.624958  5418 raft_consensus.cc:740] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:34.626209  5418 consensus_queue.cc:260] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.627262 31135 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:26:34.627672 31135 pending_rounds.cc:85] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 10
I20260709 12:26:34.627908 31135 pending_rounds.cc:107] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:26:34.628430 31135 raft_consensus.cc:2889] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.630177 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:34.634894  5109 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:34.641482  5325 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.643901  5368 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:34.644212  5325 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:34.644898  5325 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:34.645081 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:34.645738 31691 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:34.645475  5325 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:34.646473 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56407.6us
I20260709 12:26:34.627803  5418 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:34.617707  5380 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:34.595038  5106 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.656791 31140 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:34.614989  5487 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:34.609836  5152 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:34.666162 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 50495us
I20260709 12:26:34.701179 31106 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:34.654614 31141 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:34.668291 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:34.671025 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.681934 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1409) took 1565 ms. Trace:
I20260709 12:26:34.703481 31698 rpcz_store.cc:276] 0709 12:26:33.116243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.116446 (+   203us) service_pool.cc:224] Handling call
0709 12:26:33.116843 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.117400 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.117443 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.117997 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.030424 (+912427us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b180040fea0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:26:34.357255 (+326831us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:34.378462 (+ 21207us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.681259 (+302797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.681414 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.681468 (+    54us) raft_consensus.cc:1685] finished
0709 12:26:34.681564 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.681736 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26438,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":35614,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":9148,"spinlock_wait_cycles":119132416,"thread_start_us":571127,"threads_started":2,"wal-append.queue_time_us":1485429}
I20260709 12:26:34.682753  5479 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:34.706530  5479 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.709205  5479 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:34.709757  5479 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.604696  5060 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.698781 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:34.719534 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.723860 30987 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:34.725278 31138 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:34.655968  5467 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.614449  5276 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.729965  5491 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:34.725150  5479 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.739526  5467 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.742726  5368 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:34.743527  5368 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.730664  5492 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:34.751048  5492 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.751451  5492 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:34.698433  5328 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:34.753208  5328 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.758428 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:34.759693 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:34.760749  5389 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:34.761293  5389 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:34.762197  5389 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.763494  5472 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:34.764027  5472 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:34.764632  5095 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.653596  5431 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.768350  4673 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:34.726637  5152 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.771123  5491 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.771552  5491 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:34.665644  5380 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.777567 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:34.836766 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.846143 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:34.846885 31138 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:34.659668  5482 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.886663 31140 raft_consensus.cc:2393] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:34.840304  5380 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:34.703032  5378 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.765236  5095 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:34.934662 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:35.019098 31064 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.002955  5109 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:35.050379  5109 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.051097  5095 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.011073  5480 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:26:35.058485 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 496574us
W20260709 12:26:35.070122 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 310533us
W20260709 12:26:35.070143 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 135600us
I20260709 12:26:35.084391 31064 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:35.115623 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:35.116433 31693 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:35.120600 30955 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.010s	sys 0.039s
W20260709 12:26:35.121079 30955 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.010s	sys 0.039s
W20260709 12:26:35.121455 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 516123us
I20260709 12:26:35.058555  5281 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:35.156842 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.058800  5480 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:35.116636 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:35.163113 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 400987us
I20260709 12:26:35.164958 30991 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:26:35.167049 31694 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:26:35.159170 31137 consensus_meta.cc:287] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.605s	user 0.017s	sys 0.000s
I20260709 12:26:35.160710 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:35.169207 31137 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:35.083874  5496 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:35.174885 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.175745 31136 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
W20260709 12:26:35.185667 31139 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.006s
I20260709 12:26:35.186867 31139 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:35.161656  5110 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.007s	sys 0.007s
I20260709 12:26:35.158757  5281 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:35.185037 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.205762 31140 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:35.159790  5399 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.197062 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:35.204607 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:35.208648 31141 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:35.211037 31137 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:26:35.178623  5496 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:35.188045  5110 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.007s	sys 0.008s
I20260709 12:26:35.182418  5328 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.219945 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:35.230841 31143 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:35.230979 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 112782us
W20260709 12:26:35.229452 30983 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.009s
I20260709 12:26:35.231732 30983 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:35.207536  5152 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:35.232754 30988 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.011s	sys 0.057s
W20260709 12:26:35.234012 31691 consensus_meta.cc:287] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.588s	user 0.017s	sys 0.000s
W20260709 12:26:35.229055 31061 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.019s	sys 0.044s
I20260709 12:26:35.212913  5106 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:35.234508 31691 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
W20260709 12:26:35.507192 31060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.025s	sys 0.016s
W20260709 12:26:35.302904 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.263540 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:35.299497 31063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.009s	sys 0.014s
W20260709 12:26:35.555848 31060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.027s	sys 0.021s
I20260709 12:26:35.299360 31143 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
W20260709 12:26:35.268510 31142 consensus_meta.cc:287] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.700s	user 0.008s	sys 0.007s
W20260709 12:26:35.237720 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.292454 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.234452 30988 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.012s	sys 0.057s
I20260709 12:26:35.284394 31693 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:35.270697 31138 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:35.575587 31142 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:35.578056 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2884) took 1183 ms. Trace:
I20260709 12:26:35.578091 30985 raft_consensus.cc:2410] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:26:35.578564 31142 rpcz_store.cc:276] 0709 12:26:34.394360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.394482 (+   122us) service_pool.cc:224] Handling call
0709 12:26:35.577985 (+1183503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:35.579988 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.276288 31064 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:35.579449  5536 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:35.563644 31141 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
W20260709 12:26:35.345142 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 111404us
I20260709 12:26:35.581763  5536 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.233186  5109 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.584632  5536 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:35.581106 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 3184) took 1195 ms. Trace:
I20260709 12:26:35.586024 31064 rpcz_store.cc:276] 0709 12:26:34.385211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.389073 (+  3862us) service_pool.cc:224] Handling call
0709 12:26:34.933362 (+544289us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b4c002b2280. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee9b60a20 00007f2ee9b71a29 00007f2ee9b37644 00007f2ee9b5ef2d 00007f2ee9b5eb93 00007f2eeb88f18b 00007f2eeb88fcb5 00007f2eeb88af30 00007f2eeb88c7de 00007f2eeb89827c 00007f2ef0859b17
0709 12:26:35.581008 (+647646us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14278528}
W20260709 12:26:35.556391 31063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.954s	user 0.013s	sys 0.017s
W20260709 12:26:35.590626 31061 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.028s	sys 0.060s
I20260709 12:26:35.591632 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1766) took 1298 ms. Trace:
I20260709 12:26:35.592159 31061 rpcz_store.cc:276] 0709 12:26:34.293532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.300389 (+  6857us) service_pool.cc:224] Handling call
0709 12:26:34.343608 (+ 43219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.352545 (+  8937us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:34.412438 (+ 59893us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.591173 (+1178735us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:35.591259 (+    86us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:35.591332 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.591419 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.591563 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9889,"prepare.queue_time_us":53924,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":17125,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":24545,"thread_start_us":1153442,"threads_started":2,"wal-append.queue_time_us":321799}
W20260709 12:26:35.594415 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 441896us
I20260709 12:26:35.594791 31135 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
W20260709 12:26:35.442158 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 277779us
W20260709 12:26:35.299504 31033 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.045s	sys 0.058s
W20260709 12:26:35.607043 31033 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.053s	sys 0.066s
W20260709 12:26:35.601096 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.215898  5496 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:35.230469  5401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.027s	sys 0.042s
W20260709 12:26:35.400445  5511 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 12:26:35.273154  4876 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.936s	user 0.012s	sys 0.008s
W20260709 12:26:35.607499 31696 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.009s
I20260709 12:26:35.607394 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3180) took 1223 ms. Trace:
W20260709 12:26:35.400089  5504 log.cc:927] Time spent T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:26:35.288077  5430 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.954s	user 0.017s	sys 0.002s
W20260709 12:26:35.341516  5421 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.009s	sys 0.024s
I20260709 12:26:35.556906  5427 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:35.235625 30883 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.004s	sys 0.016s
I20260709 12:26:35.609969 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2787) took 1167 ms. Trace:
W20260709 12:26:35.614075 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.646473 31696 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.009s	sys 0.010s
I20260709 12:26:35.302430  5060 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:35.597182  5300 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.216s	user 0.015s	sys 0.004s
I20260709 12:26:35.669404  5522 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:35.301272  5379 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:35.263259  5328 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.670437  5328 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.305454  5325 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.671823  5325 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.672344  5325 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.672933  5328 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.673657  5328 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.674117  5328 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.674950  5325 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.258281  5378 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.258440  5467 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.266683  5472 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.676461  5328 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.646696 31060 rpcz_store.cc:276] 0709 12:26:34.384149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.396637 (+ 12488us) service_pool.cc:224] Handling call
0709 12:26:34.396902 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.397143 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:34.608511 (+211368us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.607074 (+998563us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:35.607123 (+    49us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:35.607169 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.607231 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.607341 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18046,"prepare.queue_time_us":36997,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":1102665,"threads_started":2,"wal-append.queue_time_us":965937}
I20260709 12:26:35.359982  5518 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:35.612059 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:35.250841  5142 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:35.680691  5518 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.681483 31140 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:26:35.346848  5276 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.683305  5518 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.684185 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.684041  5142 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:35.684826  5487 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.952s	user 0.011s	sys 0.014s
I20260709 12:26:35.685215  5487 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:35.665145 30883 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.006s	sys 0.016s
I20260709 12:26:35.686250 30986 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 20: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:35.287815  5482 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.563930  5399 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:35.666337  5430 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:35.686959  5399 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:35.663861 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 55609.4us
W20260709 12:26:35.667193  5491 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.895s	user 0.013s	sys 0.021s
I20260709 12:26:35.688000 30955 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:35.688105  5491 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.689254 31144 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:35.690210  5399 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.255149  5368 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.251677  5479 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.262399  5281 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.692778 31135 raft_consensus.cc:1240] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:26:35.694010  5281 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.694741  5379 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:35.695744  5379 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.670004  5522 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:35.650736 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 1.06562e+06us
I20260709 12:26:35.685329 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.270531  5480 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.698905 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:35.699064 31144 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:26:35.699407  5106 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.699973  5106 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.687306 30989 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:35.687510 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:35.646198  5401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.071s	sys 0.146s
I20260709 12:26:35.690972 30955 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:35.701674 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.661321 31106 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:35.662647  5421 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.012s	sys 0.031s
I20260709 12:26:35.705293 31143 raft_consensus.cc:2410] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 12:26:35.708050 30958 leader_election.cc:336] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Timed out: RequestConsensusVote RPC to 127.29.37.67:36565 timed out after 1.540s (SENT)
I20260709 12:26:35.664742  5389 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:35.709807 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48546.7us
I20260709 12:26:35.665216 30988 rpcz_store.cc:276] 0709 12:26:34.442871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.443010 (+   139us) service_pool.cc:224] Handling call
0709 12:26:34.445155 (+  2145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.447939 (+  2784us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:34.576180 (+128241us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.607144 (+1030964us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.609544 (+  2400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:35.609596 (+    52us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:35.609669 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.609758 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.609898 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2184,"prepare.queue_time_us":178478,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":82,"raft.queue_time_us":71202,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6866,"spinlock_wait_cycles":10537728,"thread_start_us":1095167,"threads_started":2,"wal-append.queue_time_us":1007832}
I20260709 12:26:35.710711 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:35.690433  5431 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:35.690236  5491 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:35.715372  5389 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.690621  5109 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 12:26:35.718962  5536 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:35.724583  5536 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.716288 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:35.726626  5536 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.716563  5519 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.727187  5519 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.729359  5519 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.650547  4876 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.709395  5399 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:35.718158  5482 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.713522  5060 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:35.744315 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.745117 31141 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:35.745605 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.717046  5431 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.718359  5401 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:35.727149  5536 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:35.777729 31141 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:26:35.735096  5380 consensus_meta.cc:287] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.810s	user 0.005s	sys 0.004s
I20260709 12:26:35.777565 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.778832 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.781067 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.781993 31144 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 22: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:35.732821  5142 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:35.787500 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 96643us
I20260709 12:26:35.775731  5431 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:35.775898  5491 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:35.794854 30955 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:35.794270 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.775707  5060 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:35.798570 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.797720 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:35.798700 31141 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:35.794617 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:35.800861 31144 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:35.801991 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:35.805701 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:35.806499 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.808027 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.808943 31064 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
W20260709 12:26:35.798952  5144 consensus_meta.cc:287] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 1.466s	user 0.012s	sys 0.017s
I20260709 12:26:35.811206 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:35.816738 31067 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:35.769797  5482 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:35.823766 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 157200us
I20260709 12:26:35.795447  5421 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:35.825176 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:35.798210  5472 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:35.809882  5033 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:35.799734  5142 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:35.842983 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 53303.8us
W20260709 12:26:35.818550  5418 consensus_meta.cc:287] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.146s	user 0.015s	sys 0.013s
I20260709 12:26:35.783372  5536 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.855588 31033 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:35.882511 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:35.812088  5467 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:35.853327 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.785983  5380 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:35.876858 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 168828us
W20260709 12:26:35.883559 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 167373us
I20260709 12:26:35.800226  5399 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:26:35.884359 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.906972 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:35.910996 31065 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:35.838492  5487 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.886857 31067 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:35.913033  5480 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:35.914650  5538 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:26:35.915676  5553 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:35.916270 31061 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:35.917140  5543 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:26:35.917227 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.918288  5509 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:26:35.918543 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:35.919164  5509 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:35.919363 31144 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:26:35.919607  5509 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:35.918789  5548 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:35.920558  5509 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.920991  5399 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:35.923014 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:35.922164  5399 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:35.887101 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 91923.1us
I20260709 12:26:35.924382  5399 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.926369 31106 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:35.854633  5492 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.103s	user 0.010s	sys 0.009s
I20260709 12:26:35.891237 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:35.927241 31062 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:35.927140  5492 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.929178  5492 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:35.930516 31137 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:35.930804  5300 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.931262  5300 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:35.891732 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.933180  5300 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:35.933698  5300 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:35.934015  5300 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:35.934480  5300 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.934751 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51374 (request call id 1067) took 1455 ms. Trace:
I20260709 12:26:35.934937 31691 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:35.935190 31696 rpcz_store.cc:276] 0709 12:26:34.479795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.479920 (+   125us) service_pool.cc:224] Handling call
0709 12:26:34.561983 (+ 82063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:34.919437 (+357454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:34.919662 (+   225us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:34.965600 (+ 45938us) log.cc:844] Serialized 26 byte log entry
0709 12:26:35.920483 (+954883us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b180045b140. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1
0709 12:26:35.920624 (+   141us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:35.920664 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.920729 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.934472 (+ 13743us) raft_consensus.cc:1685] finished
0709 12:26:35.934490 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.934693 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":191639,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":62,"spinlock_wait_cycles":4223360,"thread_start_us":921718,"threads_started":2,"wal-append.queue_time_us":656365}
I20260709 12:26:35.937050 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:35.937666 31694 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:35.938773 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:35.939955 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:35.940501 31698 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:35.842092  5519 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:35.886765  5142 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:35.887372  5467 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.942420  5467 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:35.946857  5540 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 12:26:35.947085 30989 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2779) took 1582 ms (client timeout 1674 ms). Trace:
I20260709 12:26:35.947391  5142 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.866683  5521 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:35.947508 30989 rpcz_store.cc:269] 0709 12:26:34.365121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.365310 (+   189us) service_pool.cc:224] Handling call
0709 12:26:35.946910 (+1581600us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54002c1354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:35.947010 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3018882432}
I20260709 12:26:35.948298  5521 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.866444  5325 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.950524  5325 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.952368 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:35.952551  5325 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.954447 31058 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:35.954591 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.955763 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:35.955917 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.956558 31058 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:35.957392 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.958478  5519 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.854259  5095 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.107s
I20260709 12:26:35.959013  5519 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.959877  5095 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:35.960330  5095 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:35.960943  5527 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 12:26:35.962261 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:35.914014 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.964257 31067 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:35.894989  5522 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.895817  5389 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:35.893921  4876 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:35.897902  5536 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:35.851163  5033 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:35.899112 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3181) took 1514 ms. Trace:
I20260709 12:26:35.900602  5482 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.986428 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.987371 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1295) took 1314 ms. Trace:
I20260709 12:26:35.903680  5528 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:35.904614  5418 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:35.850476  5549 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:35.905761  5110 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.854936  5472 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.827798  5144 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.907428  5380 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:35.907595 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:35.907822  5421 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:35.906998 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:35.909348 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:35.909623 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:35.910188 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:35.946810 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:35.893925  5060 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:35.965835 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:35.969583 31065 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:35.977145 31139 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:36.019434 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 109901us
I20260709 12:26:35.970943 31063 rpcz_store.cc:276] 0709 12:26:34.384423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.426152 (+ 41729us) service_pool.cc:224] Handling call
0709 12:26:34.453051 (+ 26899us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:34.473671 (+ 20620us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:34.567458 (+ 93787us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.806611 (+1239153us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:35.812450 (+  5839us) log.cc:844] Serialized 26 byte log entry
0709 12:26:35.812645 (+   195us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:35.812680 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.812742 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.898905 (+ 86163us) raft_consensus.cc:1685] finished
0709 12:26:35.898931 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.899056 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31998,"prepare.queue_time_us":18160,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":81320,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":74894,"thread_start_us":1187562,"threads_started":2,"wal-append.queue_time_us":1000303}
I20260709 12:26:35.937711  5106 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:36.007920 30989 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:35.986243  5033 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:35.967074  5328 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:36.016230 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:36.005893  5521 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:36.019513 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 109392us
I20260709 12:26:35.987751 30990 rpcz_store.cc:276] 0709 12:26:34.672667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.988033 (+315366us) service_pool.cc:224] Handling call
0709 12:26:35.026997 (+ 38964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.052357 (+ 25360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.052402 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:35.091566 (+ 39164us) log.cc:844] Serialized 26 byte log entry
0709 12:26:35.612360 (+520794us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:35.612397 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.612458 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.987157 (+374699us) raft_consensus.cc:1685] finished
0709 12:26:35.987179 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.987315 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15239,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":33093,"thread_start_us":415376,"threads_started":1,"wal-append.queue_time_us":529884}
I20260709 12:26:36.011618 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:36.021162 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 165654us
W20260709 12:26:36.021298 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 74560.3us
I20260709 12:26:36.007896  5549 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:36.023420 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57653.7us
I20260709 12:26:36.040131 31067 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:35.987449  5509 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:36.023509 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 114241us
I20260709 12:26:36.023196 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:36.028982 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.022790 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:36.065726 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:36.066522 31065 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:36.066807 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:36.023787 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45360 (request call id 2883) took 1625 ms. Trace:
W20260709 12:26:36.063423 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.070043 31062 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:36.073887 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:36.075138 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:36.014045  4876 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:36.022974  5536 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:36.072700 31139 rpcz_store.cc:276] 0709 12:26:34.393943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.394123 (+   180us) service_pool.cc:224] Handling call
0709 12:26:35.962336 (+1568213us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54000942d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88ba3d 00007f2ef085bcd0 00007f2ee9de0254 00007f2ee9de01e2 00007f2ee9de0111 00007f2ee9de008c 00007f2ee9ddf342 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:36.019619 (+ 57283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449889024}
I20260709 12:26:36.039561  5389 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:36.099134 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:36.041093  5517 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:36.039234  5480 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:36.103055 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:36.104348 31067 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:36.116868 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:36.118816 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:36.119079 31061 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
W20260709 12:26:36.122087 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59084.3us
I20260709 12:26:36.125460 31062 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:26:36.123586 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:36.074123  5549 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.040880  5487 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:36.079953  5430 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:36.131237 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.012040  5528 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.012580  5095 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.094270  5380 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:36.022716  5110 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.134083 31106 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:36.136058 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:36.145483 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:36.144090 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:36.151325 31067 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.113119  5521 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:36.106985  5479 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:36.117267  5517 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:36.146955 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.159472 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.130919  5378 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:36.148483 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.148887 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.149452 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.165139 31137 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:36.122910  5518 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:36.152877 30955 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:36.093851  5152 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:26:36.156173 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.224334 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.224843 31135 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
W20260709 12:26:36.225057 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 72251.1us
I20260709 12:26:36.226318 30955 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:36.157244 31694 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:36.159210 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.227734 31141 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:36.226511 30984 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:36.228679 31068 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:26:36.122697  4876 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.158403 31058 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:36.160204 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:36.233494 31061 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:36.071162  5328 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.219995  5521 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.237273  5328 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:36.237653  5521 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:36.237841 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 107924us
I20260709 12:26:36.158366 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.238350  5555 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.238569  5561 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:36.238772 31139 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:36.238996  5555 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.239718 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:36.240448 31139 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:36.110790 31661 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:36.241300  5555 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:36.163039 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:36.241876 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 131179us
I20260709 12:26:36.242027  5281 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:36.242635  5281 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
W20260709 12:26:36.242903  5496 consensus_meta.cc:287] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.001s
I20260709 12:26:36.155573  5525 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:36.241169  4673 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:36.164335  5378 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.160086 31138 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:36.225070 31144 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:36.231855 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:36.160617  5487 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:36.233717 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.231189  5546 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:36.264968 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.231431  5109 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:36.239174  5561 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.244684 31058 raft_consensus.cc:3055] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:36.222074 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 98537.8us
I20260709 12:26:36.152947  5389 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:36.280646 31137 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:36.164074  5517 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:36.247320 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.223767 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.254735 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:36.250520 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.290020 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.249168  5300 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.255777  5548 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:36.259445 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:36.302022 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.303675 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.257948  5378 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:36.306730 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47410.5us
W20260709 12:26:36.166463 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57627.2us
I20260709 12:26:36.251624 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:36.243332  5496 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.313627 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:36.257530  5487 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:36.230010  5479 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.154495  5380 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:36.351897 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 49982.1us
I20260709 12:26:36.280042 31058 raft_consensus.cc:740] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:36.294835 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.352671 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.299734  5519 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:36.262262 31067 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:36.251783  5421 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:36.281746  5379 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:36.332639 31109 connection.cc:256] RPC call timeout handler was delayed by -1.9145475159998568s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
W20260709 12:26:36.359599 31109 connection.cc:256] RPC call timeout handler was delayed by -1.6585628029997679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 12:26:36.359962 31109 connection.cc:256] RPC call timeout handler was delayed by -1.6252167170000575s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 12:26:36.360311 31109 connection.cc:256] RPC call timeout handler was delayed by -1.5475916800000959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 12:26:36.360699 31109 connection.cc:256] RPC call timeout handler was delayed by -1.4956628029999592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
W20260709 12:26:36.361140 31109 connection.cc:256] RPC call timeout handler was delayed by -1.4344628029998603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
W20260709 12:26:36.361514 31109 connection.cc:256] RPC call timeout handler was delayed by -1.378147515999899s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2392s for a response
W20260709 12:26:36.361899 31109 connection.cc:256] RPC call timeout handler was delayed by -1.3133475159997943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2464s for a response
W20260709 12:26:36.362370 31109 connection.cc:256] RPC call timeout handler was delayed by -1.1823926519996348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22870000000000001s for a response
I20260709 12:26:36.301964  5328 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:36.301482  5472 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:36.281674  5475 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:36.313397 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:36.349308  5325 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.274636  5546 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.316367 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:36.270462 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.311543 31068 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:36.347864 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:36.255822  4673 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:36.314354 30987 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:36.315855 30989 pending_rounds.cc:85] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 12
I20260709 12:26:36.357116  5522 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:36.379611 30989 pending_rounds.cc:107] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:26:36.356215  5563 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:36.315234  5106 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:36.265913  5539 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:36.357658 31142 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 22: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:36.358068 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.312291 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:36.243626  5281 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.251729  5525 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.301827  5548 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:36.266435 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:36.354627  5487 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:36.351997  5479 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.379899 30989 raft_consensus.cc:2889] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:36.354409 31106 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:36.356315  5421 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:36.380981 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.357090  5144 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:36.364966  5553 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:36.427861 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:36.379187  5546 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.387234  5106 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:36.433951 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 121764us
I20260709 12:26:36.390954  5492 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:36.451006 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:36.390986  5539 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:36.391988  5152 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:36.456034 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 66652.6us
I20260709 12:26:36.380831  5480 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
W20260709 12:26:36.454205 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 199559us
I20260709 12:26:36.457309 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:36.448261 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 93916.3us
I20260709 12:26:36.374253  5379 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.379645  5325 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.465332 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:36.449350  5421 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:36.465809  5564 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 12:26:36.456503  5152 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.456733  5539 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:36.453400  5106 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.441455  5109 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:36.592133 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 43928.2us
I20260709 12:26:36.468730  5536 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:36.556782 30988 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:36.588981  5152 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.467051  5492 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:36.604262 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 176524us
I20260709 12:26:36.549950  5276 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:36.598064 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:36.612524 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:36.613445 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:36.616909 31065 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:36.475600  5144 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.127s
I20260709 12:26:36.476146  5553 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.127s
I20260709 12:26:36.598894  5399 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:36.614455 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 153574us
I20260709 12:26:36.620551 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:36.588428  5528 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:36.621487 30987 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:36.616524 31068 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
W20260709 12:26:36.622277 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 161347us
I20260709 12:26:36.595855  5110 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:36.624677 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:36.455565  5389 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.626845 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:36.465353  4673 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.636778 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.637406 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:36.648767  5562 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 12:26:36.617499  5276 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:36.654132 30958 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:36.622202  4876 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:36.661172 30991 raft_consensus.cc:2393] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 23: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:36.661221 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:36.628485  5496 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:36.681313 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.684559 30988 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
W20260709 12:26:36.648597  5467 consensus_meta.cc:287] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.004s
W20260709 12:26:36.695737 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 59035us
W20260709 12:26:36.692906 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 82668.8us
W20260709 12:26:36.696547 31063 consensus_meta.cc:287] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.000s
I20260709 12:26:36.696866 31063 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:36.645110  5561 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:36.653986  5401 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:36.659010  5276 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.602324  5548 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.661707  5536 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:36.717392 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 596052us
W20260709 12:26:36.720579 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 72189.6us
I20260709 12:26:36.691967  5556 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:36.722664 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.723441 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:36.726068 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 78418us
I20260709 12:26:36.703869  5380 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:36.706102  5547 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.727083  5480 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:36.702381  5467 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.754035 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.738888  5556 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:36.714706  5577 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:26:36.760682 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:36.741267  5575 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:36.768181 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.768615 31136 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:36.770480 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.770623 30988 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:36.771679 31664 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.773635 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.777771 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:36.778445 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.754664  5525 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:36.786854 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.787490 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.825742  5379 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:36.770292  5575 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.772161  5528 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.777432  5518 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:26:36.843848 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 90430.7us
I20260709 12:26:36.866329 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:36.772866  5475 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.755864  5576 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.203s
I20260709 12:26:36.812733  5572 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.812911 31689 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:26:36.844379 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 124672us
I20260709 12:26:36.844790 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:36.838296  5580 consensus_queue.cc:1059] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.117s
I20260709 12:26:36.724597  5109 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.847251 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:36.759919  4673 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:36.854951 31692 raft_consensus.cc:2393] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 16: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:26:36.864837 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 168386us
I20260709 12:26:36.770027  5401 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:36.876720 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:36.779644  5325 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:36.889739 31697 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:36.891242 31695 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:36.753862  5547 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.897159 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:36.898790 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 125221us
W20260709 12:26:36.868084  5586 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:36.777279  5583 raft_consensus.cc:2764] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 12:26:36.867388  5480 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:36.865660  5575 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.898438  5518 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.917469 31136 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:26:36.891191  5528 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.903963  5418 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:36.903255  5492 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:36.921815  5518 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:36.939045  5560 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.002s	sys 0.005s
I20260709 12:26:36.902534  4673 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.934589  5379 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:36.934918 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:36.942309 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:36.938526  5506 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:36.931906  5431 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.121s	user 0.003s	sys 0.009s
I20260709 12:26:36.928890  5482 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:36.926440  5418 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:36.950616  5596 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:36.954497  5590 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:36.954639  5578 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:26:36.955052  5584 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:36.955473  5601 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:26:36.900543  5572 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.955734  4673 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:36.950906  5379 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:36.949859  5560 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.002s	sys 0.006s
I20260709 12:26:36.964481 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:36.971483 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:36.972370  5583 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:36.972712  5379 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.974077  5379 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:36.976383  5587 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 12:26:36.979452  5591 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:26:36.983695  5584 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:36.984721  5584 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.988423  5590 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:36.989452  5590 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:36.992645  5573 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.400s	user 0.003s	sys 0.000s
I20260709 12:26:36.999718 31106 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.001250  5389 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:37.001866  5389 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:37.002982  5389 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.979508  5325 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:37.005817  5325 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.006254  5325 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:37.011165  5588 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
W20260709 12:26:37.012725 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 92528us
I20260709 12:26:37.013147 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.014110 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:37.015383 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:37.016304  5496 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:37.016654  5570 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.582s	user 0.001s	sys 0.000s
I20260709 12:26:37.016973  5496 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:37.017158 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:37.017724  5496 consensus_queue.cc:1059] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:37.018244  5568 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.775s	user 0.000s	sys 0.011s
I20260709 12:26:37.019573  4876 raft_consensus.cc:3055] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:37.020020  4876 raft_consensus.cc:740] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:37.021112  4876 consensus_queue.cc:260] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.910404  5556 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:36.973162  5518 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.025868 31058 consensus_queue.cc:260] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.982007  5300 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.027493 31058 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.030957  4876 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:37.032403  5430 consensus_meta.cc:287] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.890s	user 0.009s	sys 0.000s
I20260709 12:26:37.032542 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41814 (request call id 1304) took 1080 ms. Trace:
I20260709 12:26:37.032806 30986 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:37.032933 30984 rpcz_store.cc:276] 0709 12:26:35.951659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.961981 (+ 10322us) service_pool.cc:224] Handling call
0709 12:26:37.032367 (+1070386us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54001f7054. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:37.032484 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606342528}
I20260709 12:26:37.035215 31058 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:26:37.035928  5378 consensus_meta.cc:287] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.684s	user 0.008s	sys 0.000s
I20260709 12:26:37.036743 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1773) took 1145 ms. Trace:
W20260709 12:26:37.038800 31142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 1303) took 1779 ms (client timeout 1734 ms). Trace:
W20260709 12:26:37.039268 31142 rpcz_store.cc:269] 0709 12:26:35.259263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.579670 (+320407us) service_pool.cc:224] Handling call
0709 12:26:37.038562 (+1458892us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b5400062a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:37.038716 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786423680}
I20260709 12:26:37.040227  5152 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.041760 30983 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:26:37.042550  5380 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
W20260709 12:26:37.043788  5558 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.770s	user 0.000s	sys 0.002s
I20260709 12:26:37.044296  5528 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.037204 31064 rpcz_store.cc:276] 0709 12:26:35.891314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.891601 (+   287us) service_pool.cc:224] Handling call
0709 12:26:35.891924 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:35.963312 (+ 71388us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:36.147399 (+184087us) log.cc:844] Serialized 14 byte log entry
0709 12:26:36.279443 (+132044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:36.279517 (+    74us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:36.279588 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.988118 (+708530us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800413ce0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:37.010517 (+ 22399us) spinlock_profiling.cc:243] Waited 7.49 ms on lock 0x7b1800413ce0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87752a6 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419
0709 12:26:37.015873 (+  5356us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:37.036482 (+ 20609us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b54003967d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:37.036542 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.036679 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7950,"prepare.queue_time_us":315206,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":18367,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":673788288,"thread_start_us":248448,"threads_started":2,"wal-append.queue_time_us":224483}
I20260709 12:26:37.043659  5492 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.045677  5492 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:37.055230  5567 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.814s	user 0.000s	sys 0.004s
I20260709 12:26:36.959452  5482 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:37.059129  5482 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.060123  5325 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.060468  5482 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.962277  5506 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.959662  5418 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.075579  5519 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.076229  5519 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:36.960228  5536 raft_consensus.cc:2764] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:26:37.078608  5060 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:37.079254  5060 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.081017  5276 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.091496 30988 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:37.092263  5579 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.369s	user 0.002s	sys 0.000s
I20260709 12:26:36.963415  5431 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.081801  5060 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:37.099458  5571 log.cc:927] Time spent T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.497s	user 0.002s	sys 0.000s
I20260709 12:26:37.115216  5431 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:36.961381  5401 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:37.090462  5033 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.016s	user 0.008s	sys 0.003s
I20260709 12:26:37.119402  5033 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.121902  5033 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.089314  5106 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:37.084545  5479 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.621s	user 0.007s	sys 0.001s
I20260709 12:26:37.084049  5095 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:36.964231  5467 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.062121  5325 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.036324  5378 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:37.108997 30876 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2439e7934b98446ca4e7d82381e8a499 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e9cccf7eb2a545ef8c2169b67ff261bf is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:26:37.110580  5574 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.477s	user 0.000s	sys 0.001s
I20260709 12:26:37.112143  5547 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.114182 30985 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:37.185043 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.185796 31065 raft_consensus.cc:2393] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:37.186246 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:37.123080  5487 consensus_meta.cc:287] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.655s	user 0.013s	sys 0.000s
W20260709 12:26:37.124070  5549 consensus_meta.cc:287] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.961s	user 0.000s	sys 0.011s
W20260709 12:26:37.125978  5509 consensus_meta.cc:287] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.046s	user 0.012s	sys 0.001s
W20260709 12:26:37.126794  5546 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.000s
I20260709 12:26:37.087172  5548 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.199510 31696 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:37.135144  5276 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.135382  5572 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.136428  5492 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.136829  5506 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.184994  5583 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:37.144901  5496 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:37.187448  5399 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:37.032843  5430 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:37.175973 31064 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:37.180599 31109 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:37.104046 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.183027 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.220353 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.221560 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:37.184335 30876 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 75540.6us
I20260709 12:26:37.186538 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:37.187309 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 215920us
I20260709 12:26:37.222658 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:37.188887 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 312787us
I20260709 12:26:37.137681  5519 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:37.164063 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 259732us
I20260709 12:26:37.229347  5605 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.229843  5605 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.230414 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.231081 31143 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:37.231421 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.231966 31135 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:37.232203  5605 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.231053 31142 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:37.232795 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.233379 31141 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:37.234548 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.235563 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.219256  5594 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:26:37.236524 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.237497 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.237949 31063 raft_consensus.cc:2393] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 22: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:37.238827 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:37.239459  5421 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.001s
I20260709 12:26:37.187875  5401 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.182088  5479 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.240008 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.240852 31058 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:26:37.240039  5421 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.242079 30987 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:37.223243 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.243067 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.243381  5479 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.243700  5421 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.244263  5596 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.244105 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.245312 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.245791 31067 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:37.232251 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.247475 31139 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:37.247938 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.223253 31064 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:37.223634 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:37.239539 31059 raft_consensus.cc:2393] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 24: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:37.248793 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.189455  5467 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.190093  5487 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.217011  5399 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:37.192327  5549 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.244104  5401 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.205116  5546 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.382726 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:37.256322 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.383570 30983 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 15.
I20260709 12:26:37.384164 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:37.353317  5539 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.000s
I20260709 12:26:37.385072  5539 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.386015 30992 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:37.387002  5539 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.256150  5325 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:37.387698  5325 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.387959  5325 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:37.251886  5580 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.389189  5580 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.389551  5580 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.273330 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:37.317279 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 3068) took 1031 ms. Trace:
I20260709 12:26:37.391932 30985 rpcz_store.cc:276] 0709 12:26:36.286121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.286279 (+   158us) service_pool.cc:224] Handling call
0709 12:26:37.317043 (+1030764us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b5400227254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:37.317208 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700093696}
I20260709 12:26:37.393695 30991 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
W20260709 12:26:37.394871  5517 consensus_meta.cc:287] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 1.080s	user 0.005s	sys 0.006s
I20260709 12:26:37.395506  5517 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.396534 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:37.397119  5418 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.397302 31694 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:37.397619  5517 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.240514  5583 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.398399 31693 raft_consensus.cc:2393] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:37.395816  5152 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:37.398838  5475 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.508s	user 0.005s	sys 0.004s
I20260709 12:26:37.395503  5487 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.399281  5475 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:37.400058 31697 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:37.295085  5106 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.357321  5492 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:37.402084  5418 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.402405  5492 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:37.311374  5378 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.293968  5572 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.362160 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:37.402803 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:37.294299 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.259408  5467 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:37.404506 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 229291us
I20260709 12:26:37.405475  5572 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.412223 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:37.414253 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 156806us
W20260709 12:26:37.391487  5574 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 12:26:37.291590  5522 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.904s	user 0.010s	sys 0.001s
I20260709 12:26:37.410712  5033 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:37.405061  5399 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.418711 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:37.256009  5421 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:37.248252  5596 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:37.254498  5521 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:37.419728 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 164321us
I20260709 12:26:37.420401 31144 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:37.436949 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.437325 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.438133 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.438493 31063 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:37.440688  5033 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.441928 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.425480  5547 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:26:37.443171 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.403384  5492 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.423171 31060 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:37.418413  5106 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:37.427539 30983 raft_consensus.cc:1240] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:37.427634  5611 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:26:37.428754 30987 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 18. Current term is 20. Ops: []
I20260709 12:26:37.421727  5389 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.424109  5553 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.430500 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.451856 30986 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:37.430588 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.431829 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.452627 31064 raft_consensus.cc:2393] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:37.452901 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.444555 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 3041) took 1534 ms. Trace:
I20260709 12:26:37.453686 31142 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 24: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:37.453433  5389 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.453892 31062 rpcz_store.cc:276] 0709 12:26:35.910112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.143328 (+233216us) service_pool.cc:224] Handling call
0709 12:26:36.161878 (+ 18550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:36.231713 (+ 69835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:36.231745 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:36.250122 (+ 18377us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.394075 (+143953us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:36.440305 (+ 46230us) log.cc:844] Serialized 14 byte log entry
0709 12:26:36.587606 (+147301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.587707 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:37.101319 (+513612us) spinlock_profiling.cc:243] Waited 8.5 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:37.444241 (+342922us) raft_consensus.cc:1685] finished
0709 12:26:37.444345 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.444490 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69846,"prepare.queue_time_us":42770,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":249620,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":22473,"spinlock_wait_cycles":19549952,"thread_start_us":259062,"threads_started":2,"wal-append.queue_time_us":306387}
I20260709 12:26:37.455632  5389 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.456015 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.456542 31062 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:37.451160  5553 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:37.457103  5553 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.459514 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.459883  5033 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.460969 31137 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:37.462024 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.463194  5601 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:37.463734  5601 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.255227  5594 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.170s
I20260709 12:26:37.443732 30989 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:37.474056 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.436612  5521 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:37.426729  5522 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:37.450726 31693 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51374 (request call id 1074) took 1203 ms. Trace:
W20260709 12:26:37.453109  5575 consensus_meta.cc:287] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.002s
I20260709 12:26:37.452064 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.482076 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.464720 31060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51942 (request call id 3197) took 1218 ms. Trace:
I20260709 12:26:37.483105 31060 rpcz_store.cc:276] 0709 12:26:36.246118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.246316 (+   198us) service_pool.cc:224] Handling call
0709 12:26:37.464567 (+1218251us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b54002a4c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:37.464666 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2266626432}
I20260709 12:26:37.453528 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:37.421826 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 193146us
I20260709 12:26:37.431177  5421 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.443348 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.471563 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.450096 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.474779 31142 raft_consensus.cc:2368] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21.
I20260709 12:26:37.486094 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:37.487749 30986 raft_consensus.cc:2410] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:26:37.488569 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.490115  5601 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.485206  5575 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.491335 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.491875 31139 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.492805  5575 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.436506  5596 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.476346 31693 rpcz_store.cc:276] 0709 12:26:36.246957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.247099 (+   142us) service_pool.cc:224] Handling call
0709 12:26:37.450530 (+1203431us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540055c654. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:37.450643 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2550931328}
I20260709 12:26:37.478155 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2908) took 1224 ms. Trace:
I20260709 12:26:37.481452 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.496628 31144 rpcz_store.cc:276] 0709 12:26:36.253950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.254077 (+   127us) service_pool.cc:224] Handling call
0709 12:26:36.288417 (+ 34340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.436915 (+1148498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.477864 (+ 40949us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001e9bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:37.478084 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2602790656}
I20260709 12:26:37.499147 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.499382 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.499966 31136 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.501446 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:37.483693 31062 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:37.448625  5106 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:37.504822 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:37.505141  5106 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.505384 31695 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.485697 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.507489 31067 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:37.507949 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.508355 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:37.509078 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.509539 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.510097 30989 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:26:37.510471 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.510593 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.510844  5033 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:37.512321 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.512475 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.513124 30983 raft_consensus.cc:2393] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:26:37.475512 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 44067.2us
I20260709 12:26:37.516083  5572 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.517409 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:37.493783 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 43763.3us
I20260709 12:26:37.485488 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.494163 30985 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:37.535971  5106 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.502475 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.503125 31140 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:37.503388 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.504148 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:37.504559 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:37.510541 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.540084 31067 raft_consensus.cc:2468] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:37.529239 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 3070) took 1221 ms. Trace:
I20260709 12:26:37.531179 31068 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:26:37.482401  5521 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.486402  5418 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:37.535105  5467 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.537313  5401 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.502526  5496 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:37.493296  5421 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.539965  5549 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.540165 31061 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 12:26:37.544028 30984 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:37.544982 31064 raft_consensus.cc:1240] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:37.544816  5575 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:37.531648 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.547948 31696 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:37.528044 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.590528 31065 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:26:37.485905 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.475075  5546 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.494678 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:37.552219 31062 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:37.555922 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.559154 31068 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:37.559090  5594 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:37.545734 30992 rpcz_store.cc:276] 0709 12:26:36.308301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.308457 (+   156us) service_pool.cc:224] Handling call
0709 12:26:36.353413 (+ 44956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:37.386398 (+1032985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.529000 (+142602us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400023ad4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:37.529178 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2374166272}
I20260709 12:26:37.533703  5325 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.576514  5547 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:37.584658  5496 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:37.565886 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.619361 31140 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:37.567741 31693 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:37.567193 31136 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:37.560261 31661 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:26:37.620833 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60670.1us
I20260709 12:26:37.621827 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:37.561702 30958 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:37.570384 31139 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:37.582950 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:37.587793 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:37.603263 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 117377us
I20260709 12:26:37.554898 31109 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:37.623785 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68946.4us
I20260709 12:26:37.599401 31698 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:37.566921  5482 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:26:37.625269 31138 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:37.568077  5389 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:26:37.627633 31064 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:26:37.624447 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.588958  5553 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.607028  5575 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:37.579460  5418 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.563743 31695 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:37.619851  5496 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:37.606877  5594 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:37.643093 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.631242  5547 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:37.638729 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.642000 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:37.638744 31065 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.630164  5517 raft_consensus.cc:2804] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:37.645802 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:37.649536  5594 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:37.659564  5517 raft_consensus.cc:697] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:37.659749 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.646572 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.660992 30986 raft_consensus.cc:2410] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:26:37.660422  5517 consensus_queue.cc:237] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.647138 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.644099 30990 raft_consensus.cc:2410] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:26:37.648748 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:37.660441 30991 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:37.666653  5575 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:37.682863  5619 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:37.686043  5580 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.690959 31061 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:37.629549  5601 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:26:37.652067 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 90470.5us
I20260709 12:26:37.702512 31063 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:37.647745 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.702740 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:37.648263 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:37.635337  5378 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:37.688608  5580 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.656142  5590 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:37.657413  4876 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.657821  5475 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:37.658927 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:37.696810  5553 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:37.696884  5325 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.701156  5596 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.726708 31060 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:37.727905  5596 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.728360  5601 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.728888  5601 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:37.711871  5572 consensus_queue.cc:1059] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:26:37.735671  5572 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:37.736025  5572 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:37.736968  5572 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.707103  5378 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.739393  5378 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:37.744225  5606 consensus_queue.cc:1059] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:37.719498  5475 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.747012  5475 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:37.751063  4876 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.751557  5590 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:37.752874  5590 raft_consensus.cc:3055] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:37.753214  5590 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:37.701581  5418 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:37.696825 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 136757us
I20260709 12:26:37.644508  5546 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:37.655335 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:37.709869 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.710950 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:37.654422  5547 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.717813 31065 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:26:37.777237 31065 pending_rounds.cc:85] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 10
I20260709 12:26:37.777513 31065 pending_rounds.cc:107] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:26:37.777745 31065 raft_consensus.cc:2889] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.718869 30992 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:37.759658 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:37.725422  5496 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.788761 31068 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:37.777655 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.789990 31062 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:37.715538  5431 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:37.725345  5594 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:37.781495 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 126255us
W20260709 12:26:37.797747 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 86884.8us
I20260709 12:26:37.802989 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:37.760867  5572 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.811156 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:37.761796  5546 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.820557 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:37.780246  5547 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.822618  5608 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:37.753926  5590 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:37.802170  5584 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.826309  5482 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:37.830981  5095 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:37.832247  5496 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.846813  5546 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:37.848582 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:37.788498  5418 raft_consensus.cc:2764] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:37.832260  4876 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.804356  5528 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:37.832402  5389 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.859836  5590 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:37.865211 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.855412  5584 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.854290  5597 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:37.878119 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:37.880432 31695 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 15.
I20260709 12:26:37.856693  5482 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.860096  5152 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:37.857481  5106 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:37.859472  5095 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.874881  5487 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:37.879231  5584 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.884025  5482 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.901242 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.901435 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.906325 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:37.901983 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.907248 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:37.907996 31139 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:37.885632  5152 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:37.906250 31136 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:37.890583  5095 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.907953 31142 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:37.910516 31138 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:37.912170 31140 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:37.914646 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:37.876713  5431 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
W20260709 12:26:37.913236 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 342602us
I20260709 12:26:37.883641  5389 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.882310  5528 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:37.934840 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.946172 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:37.949672 31693 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:26:37.970285 31693 pending_rounds.cc:85] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 12
I20260709 12:26:37.970556 31693 pending_rounds.cc:107] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:26:37.970387 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:37.970801 31693 raft_consensus.cc:2889] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.952800 31142 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:37.960397  5389 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:37.961334  5623 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:26:37.967249 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 143414us
I20260709 12:26:37.929903  5421 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.981861 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:37.985353 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 174294us
I20260709 12:26:37.984817 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:37.990538 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44485us
I20260709 12:26:37.992590 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:37.971609  5583 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:37.967806  5152 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:38.112556 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 120000us
I20260709 12:26:38.011193 31106 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:38.114375 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 103252us
W20260709 12:26:38.102243 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45681.5us
I20260709 12:26:38.008538  5421 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:38.123178 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:37.991997  5379 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:38.005733  5475 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.979161  5109 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
W20260709 12:26:38.142503 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 156415us
I20260709 12:26:38.135867  5601 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:37.996297  5528 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:37.992424  5553 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:38.056167  4876 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.101s
I20260709 12:26:37.999867  5556 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.162s
I20260709 12:26:38.143805  5106 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:38.153004  5152 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:38.113504  5389 consensus_queue.cc:1059] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 12:26:37.996167  5492 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:38.171571  5496 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:26:38.211679 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:38.186208  5640 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:38.212607 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 97445.1us
W20260709 12:26:38.214329  5521 consensus_meta.cc:287] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.000s
I20260709 12:26:38.213905 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:38.224900 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.229053 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:38.102586 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 110428us
I20260709 12:26:38.222159 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.222807 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:38.229904 31058 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:38.231568 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:38.233901 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 120475us
I20260709 12:26:38.235196 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:37.999383  5594 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:38.220705  5640 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.226055  5521 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.220170  5492 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:38.249562 30958 leader_election.cc:400] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21.
I20260709 12:26:38.240237 31106 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:38.242142  5496 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:38.263501  5496 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:38.263940  5496 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:38.231076  5106 raft_consensus.cc:3055] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:38.266324  5106 raft_consensus.cc:740] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:38.267118 31058 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:38.241948  5401 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:38.269981  5606 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:38.270310  5492 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:38.198421  5584 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.274806  5496 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.275169  5594 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:38.275707  5594 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.276401  5584 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:38.237231  5639 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:38.277884  5487 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:38.278309  5487 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:38.278375 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:38.277923  5594 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:38.279191  5487 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.279953  5639 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:38.280475  5639 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:38.243418  5640 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:38.282946 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:38.283697 31060 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:38.284396  5521 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:38.245070  5601 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:38.284886 30958 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:38.288844 31068 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
W20260709 12:26:38.291491 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56400.8us
I20260709 12:26:38.281351  5639 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.283697  5608 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:38.295240  5575 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:38.295960  5575 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:38.296914  5575 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:38.297235  5575 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:38.297485  5643 log.cc:927] Time spent T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.003s
I20260709 12:26:38.280635  5487 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:38.280716  5475 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:38.309708 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:38.288223  4876 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:38.241602  5553 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:38.318007 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 121989us
I20260709 12:26:38.313982  5539 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:26:38.312279  5606 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.322489 30881 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 14 to 16, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 16 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.322508  5606 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.298524  5106 consensus_queue.cc:260] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.327064 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:38.327666 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 225506us
I20260709 12:26:38.332068 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.314472  5556 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:38.312547  4876 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.333575 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.325248 30881 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:38.316279  5590 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:38.301328  5594 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:26:38.369850 31060 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:38.308239  5597 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:26:38.297995  5575 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.371058 31064 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:26:38.340559 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:38.308408  5482 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.372015 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.364573 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:38.382880 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.378731  4876 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:38.385694 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:38.383632 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:38.326395  5379 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:26:38.378260  5650 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.392989 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.412562 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.373570  5517 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:38.377947  5106 raft_consensus.cc:3060] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:38.417465 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:38.415783 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.421224 31140 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 22: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:38.341532  5556 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:38.418983 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.386636  5521 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:38.426206 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.429279 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:38.407255  5399 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.968s	user 0.011s	sys 0.001s
I20260709 12:26:38.424449 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:38.408890 30992 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:38.336486  5095 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.408561  5325 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:38.426160  5556 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:38.329473  5547 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.426575  5521 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:38.362950  5583 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:38.336647  5630 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.089s
I20260709 12:26:38.371722  5475 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:26:38.453577 31140 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:26:38.425568  5431 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:38.384258  5572 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:26:38.421339  5549 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:38.420390  5517 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:38.431056 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.482558 31064 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 25.
I20260709 12:26:38.422902  5636 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:38.485469 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.525249 31138 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:38.488502 30990 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:38.512347  5521 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:38.418160  5378 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.675s	user 0.007s	sys 0.004s
I20260709 12:26:38.530432  5378 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.492201 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:38.512552 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.532080 31141 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:38.533119  5378 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:38.521680  5651 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:38.533980  5651 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:38.534471  5651 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:38.493506 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:38.512820 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:38.538545 31689 raft_consensus.cc:1240] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:38.538709 30984 raft_consensus.cc:3060] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:26:38.539098  5606 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:38.424151  5596 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:26:38.526072  5636 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:38.540788  5636 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.544092  5575 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:38.546717 30883 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 201) took 49580 ms (client timeout 15000 ms). Trace:
W20260709 12:26:38.548381  5629 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:26:38.552219  5496 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:38.552840  5496 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:26:38.554572  5641 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:26:38.555146  5640 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.451980  5399 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.561781  5399 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:38.491030 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:38.563293 31059 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:38.564461 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51946 (request call id 3055) took 1342 ms. Trace:
I20260709 12:26:38.564819 31059 rpcz_store.cc:276] 0709 12:26:37.221967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.305546 (+ 83579us) service_pool.cc:224] Handling call
0709 12:26:37.305830 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.563663 (+1257833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.564253 (+   590us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400268c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:38.564406 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2891467136}
W20260709 12:26:38.547259 30883 rpcz_store.cc:269] 0709 12:25:48.965981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.019588 (+ 53607us) service_pool.cc:224] Handling call
0709 12:26:38.546615 (+49527027us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:35.395050 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:35.395748 (+   698us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:35.395846 (+    98us) write_op.cc:435] Acquired schema lock
0709 12:26:35.395869 (+    23us) tablet.cc:662] Decoding operations
0709 12:26:35.396525 (+   656us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:35.396617 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:35.396649 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:26:35.397113 (+   464us) tablet.cc:701] Row locks acquired
0709 12:26:35.397160 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:26:35.397332 (+   172us) write_op.cc:270] Start()
0709 12:26:35.397475 (+   143us) write_op.cc:276] Timestamp: P: 1783599995397257 usec, L: 0
0709 12:26:35.397512 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:26:35.398210 (+   698us) log.cc:844] Serialized 1293 byte log entry
0709 12:26:36.107471 (+709261us) op_driver.cc:464] REPLICATION: finished
0709 12:26:36.989376 (+881905us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b1800020080. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:36.989615 (+   239us) write_op.cc:301] APPLY: starting
0709 12:26:36.989727 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:36.992720 (+  2993us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:36.992774 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:37.113699 (+120925us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b1800020080. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb5d30ec 00007f2eeb61fc0a 00007f2eeb61f4f7 00007f2eeb61f490 00007f2eeb5d6f48 00007f2eef2eb1bb 00007f2eef3325d4 00007f2eef1de8da 00007f2eef1e13cb
0709 12:26:37.127118 (+ 13419us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:37.127482 (+   364us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 12:26:37.127566 (+    84us) write_op.cc:312] APPLY: finished
0709 12:26:37.127905 (+   339us) log.cc:844] Serialized 41 byte log entry
0709 12:26:37.405674 (+277769us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:37.405777 (+   103us) write_op.cc:454] Released schema lock
0709 12:26:37.405845 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:37.436189 (+ 30344us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:37.445661 (+  9472us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2199155,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":768818,"apply.run_cpu_time_us":31227,"apply.run_wall_time_us":424512,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16673,"num_ops":3,"prepare.queue_time_us":722474,"prepare.run_cpu_time_us":32734,"prepare.run_wall_time_us":446748,"raft.queue_time_us":190529,"raft.run_cpu_time_us":9991,"raft.run_wall_time_us":1041030,"replication_time_us":623040,"spinlock_wait_cycles":41089280,"thread_start_us":2346567,"threads_started":5,"wal-append.queue_time_us":706766,"wal-append.run_cpu_time_us":24695,"wal-append.run_wall_time_us":2124818}]]}
I20260709 12:26:38.568001 31141 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 25.
I20260709 12:26:38.519245 31696 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:38.476895  5418 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:38.542085  5636 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:38.568534  5033 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:38.525566  5517 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:38.420499  5650 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.572638 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:38.585795 31064 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:38.573031 30984 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 25.
I20260709 12:26:38.559305  5547 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:38.573374 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.591990 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.575460 31138 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:26:38.516482  5431 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:38.588975 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 249637us
W20260709 12:26:38.588573 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 181773us
W20260709 12:26:38.593204 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.585171  5418 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.592957 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:38.601557 31135 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
W20260709 12:26:38.601864 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 282425us
I20260709 12:26:38.646283 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.647034 31065 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:38.655421  5389 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:38.431735 30881 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.627228  5584 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:38.607620  5572 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:38.662952 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:38.668234 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:38.680447 30984 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:26:38.704300 30992 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:38.681752 31065 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:38.658710  5482 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:38.663765  5645 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 12:26:38.691931 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 92570.9us
I20260709 12:26:38.668850  5399 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:38.670640  5647 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.699069 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.701382 31138 raft_consensus.cc:1240] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:38.701481 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:38.705513 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:38.715732 30990 raft_consensus.cc:2410] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:26:38.720785 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:38.721451 31689 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:38.722151 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:38.726310 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:38.704860 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 285608us
I20260709 12:26:38.706327 31106 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:38.712090 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.727270 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.728482 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.702965  5584 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.731171 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:38.731415 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:38.706118  5572 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:38.732249 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.736543 30985 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:38.713371  5095 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:38.689090  5661 consensus_queue.cc:1059] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 12:26:38.731971  5648 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 12:26:38.679214  5431 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.713322  5399 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:38.835182 30990 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:38.835705  5556 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.837728 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:38.838769 31693 raft_consensus.cc:1240] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:38.840927 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:38.785910  5655 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:26:38.786163 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:38.786499 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:38.778805  5496 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:38.858206 31689 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:38.787889 30881 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:38.862718 31691 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: [20.12-23.13]
I20260709 12:26:38.735054  5630 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:38.866829  5528 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:38.867205  5528 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:38.867472  5399 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.868150  5528 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.792546  5418 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:38.781682  5608 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:38.869271  5608 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:38.870352  5630 raft_consensus.cc:476] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Not starting pre-election -- already a leader
I20260709 12:26:38.858467 31697 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 20: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:38.871012 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.870496  5418 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:38.871867 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:38.871955  5418 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.872960 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:38.873521 31140 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:38.873847 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:38.860941 30881 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 14 to 18, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.874504 31144 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:38.875367 30881 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 14 to 17, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 17 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.877101 30881 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.873971  5418 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:38.878196 31140 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:26:38.869264  5399 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:38.879007  5572 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.879717 31140 raft_consensus.cc:1240] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:38.737298  5584 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:38.881873 31140 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:38.883342 31137 raft_consensus.cc:2410] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 12:26:38.803171  5660 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:26:38.805748 31694 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 12:26:38.806403  5095 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.886662 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 45841.1us
I20260709 12:26:38.891567 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:38.812332  5662 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:38.816102 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:38.818990  5109 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.806421 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:38.891636 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:38.806780 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:38.827654 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:38.841399  5651 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.842360 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:38.843303  5641 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:26:38.843797  5635 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:26:38.844396  5623 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:26:38.846760 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 111092us
I20260709 12:26:38.848577  5676 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:38.822971  5640 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:38.850505 31060 raft_consensus.cc:1240] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:26:38.851442  5677 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.800848  5152 consensus_meta.cc:287] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.001s
I20260709 12:26:38.880081  5572 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:38.726506  5647 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:38.802383  5546 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.925s	user 0.008s	sys 0.004s
I20260709 12:26:38.886984 31061 raft_consensus.cc:1275] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:26:38.898289  5553 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 12:26:38.728756  5389 raft_consensus.cc:3055] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:38.895736 30992 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:38.897032 31692 raft_consensus.cc:2393] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 25: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:26:38.900970 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 84990.5us
I20260709 12:26:38.932090 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:38.908224 31144 raft_consensus.cc:2468] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
W20260709 12:26:38.891156  5492 consensus_meta.cc:287] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.000s
I20260709 12:26:38.910826 31059 raft_consensus.cc:1240] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:38.914275 31664 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.885586  5665 raft_consensus.cc:2804] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:38.911897 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.920074  5601 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:38.946205 31033 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:38.886668  5657 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.849934  5521 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:38.925523 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.949179 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:38.890801  5674 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:38.934448 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 202252us
I20260709 12:26:38.896584  5662 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:38.942893 31696 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:38.955660 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.915939 31142 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:38.944832 31664 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:38.921124  5379 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:38.930805  5389 raft_consensus.cc:740] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:38.898916  5095 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.897130  4876 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.942824  5650 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:38.901647  5109 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:38.934448  5640 consensus_queue.cc:1059] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:26:38.943861  5553 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:38.962072 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:38.953684  5325 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:38.970079  5482 raft_consensus.cc:2804] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:38.970729  5659 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:38.995879 31696 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:38.957581  5664 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:38.935601  5418 consensus_queue.cc:1048] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:26:38.927610  5546 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.945501  5492 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.018041 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.958501  5657 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:38.976058  5521 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:38.946658  5601 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:39.035290 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 79662.2us
I20260709 12:26:38.948259  5653 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.953197  5580 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.928480  5669 consensus_queue.cc:1059] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 12:26:38.968881 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:38.988267  5095 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.040119 31135 raft_consensus.cc:2368] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21.
I20260709 12:26:38.977682  5487 consensus_queue.cc:1059] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:26:38.953367  5665 raft_consensus.cc:697] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:38.971431  5661 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:39.005906 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 43907.1us
I20260709 12:26:38.962455  5379 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:38.966737  5539 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:38.963105  5561 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:39.011153 31063 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:38.922626  5677 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.030136 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:38.954185  5517 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.016750  5482 raft_consensus.cc:697] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:38.985886  5684 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:39.038007 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 80865us
I20260709 12:26:38.990429  5630 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:39.012406  5596 raft_consensus.cc:2764] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:26:39.043054 31141 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:39.045403 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 76587.7us
I20260709 12:26:39.047158 31033 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:39.032409 31064 raft_consensus.cc:1240] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:39.035944  5547 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:39.028694  5640 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:38.982753  5583 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:26:39.093667  5689 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
W20260709 12:26:39.110033 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 66091.1us
I20260709 12:26:39.103116  5647 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.110903  5546 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:39.091391  5651 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:39.092710  5683 log.cc:927] Time spent T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:26:39.093380  5681 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:39.048570  5379 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:39.115082  5431 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.092020  5584 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.117410  5584 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.051896  5669 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:26:39.107877 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.077539 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.049809  5674 raft_consensus.cc:3055] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:39.121575  5674 raft_consensus.cc:740] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:39.102739  4876 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:39.095175  5677 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:39.105940  5109 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.122558  5674 consensus_queue.cc:260] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.104471  5517 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.097438  5665 consensus_queue.cc:237] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.123876  5674 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:39.124797  5673 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 12:26:39.048342  5561 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:39.126405  5561 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:39.106525  5688 consensus_queue.cc:1059] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:26:39.126087  5482 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.127063  5669 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:39.098466  5492 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:39.128901  5688 raft_consensus.cc:3055] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:39.129281  5688 raft_consensus.cc:740] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:39.037779  5601 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:39.115121 30958 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.050122  5661 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:39.139137  5672 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.292s	user 0.003s	sys 0.000s
I20260709 12:26:39.139787  5389 consensus_queue.cc:260] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.141088  5389 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.141402 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.141638  5661 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:39.142000 31140 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:39.143133  5657 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.144373 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:39.145167 31137 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:39.146867  5659 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:39.147414  5659 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:39.067914 31138 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:39.105726 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.080147 31695 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 19: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:39.072715 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.152863 31140 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:39.119313 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:39.149878 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 102816us
W20260709 12:26:39.165491 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 46256.6us
I20260709 12:26:39.173333 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:39.172024 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 99378.9us
I20260709 12:26:39.174120 31061 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:39.148301  5659 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.180634 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.121680  5640 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.189337 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.191988 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:39.196148 31058 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:39.197552 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:39.200663 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:39.201442 31697 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 16.
I20260709 12:26:39.201531 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.203269 31138 raft_consensus.cc:2468] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:39.069857  5650 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.193534 30992 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:26:39.182431  5690 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
I20260709 12:26:39.230564 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.066959  5684 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.233122  5684 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:39.234064  5601 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:39.234452 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 46487.4us
I20260709 12:26:39.235499 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:39.236428  5601 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.237090  5601 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:39.237015 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:39.238721  5647 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.239588 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.239909 30955 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.241005 30990 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:26:39.241721  5109 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:39.242075 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.221907  5539 consensus_queue.cc:1059] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:26:39.242470  5401 consensus_queue.cc:1059] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:39.243233  5539 raft_consensus.cc:3055] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:39.243620  5539 raft_consensus.cc:740] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:39.243810 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.244417  5539 consensus_queue.cc:260] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.245134  5106 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:39.245548  5106 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:39.245679  5539 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:39.246419  5106 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.043651  5590 consensus_queue.cc:1059] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.139s
I20260709 12:26:39.249029 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:39.222774 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.130074  5688 consensus_queue.cc:260] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.229457 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.253533 30984 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:39.177471  5517 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:26:39.251798 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.262748 30983 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:39.221271 31141 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:39.215677 31058 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:39.264003 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.255396 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:39.264643 30991 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.257889 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.258671 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.265714 31136 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.182766  5492 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:39.260813 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.177716  5399 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:39.273339 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.273829 31693 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:39.274019 30992 raft_consensus.cc:2368] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:26:39.280815 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:39.242118  5109 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.010149 30881 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 11 to 15, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 15 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.282315 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.282701 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:39.289462 31697 raft_consensus.cc:2393] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:39.291182 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:39.278369  5655 log.cc:927] Time spent T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:26:39.309221 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 69376.2us
I20260709 12:26:39.310779 31691 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 19: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:39.300333 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.271962  5553 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:39.265805  5647 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:39.300621 31136 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:39.280838  5667 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.287750  5590 raft_consensus.cc:3055] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:39.322322 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.264001  5601 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.046s
W20260709 12:26:39.247996  5575 consensus_queue.cc:1175] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" responder_term: 16 status { last_received { term: 16 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 16 index: 11 } }
I20260709 12:26:39.248332  5584 consensus_queue.cc:1059] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:26:39.293372  5109 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:39.327586 30983 raft_consensus.cc:2468] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 24.
I20260709 12:26:39.287416  5399 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:39.337077 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 112850us
I20260709 12:26:39.311383 31059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:39.308561  5572 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:39.320098  5033 raft_consensus.cc:3055] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:39.338804 31664 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:39.350912 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:39.330072  5590 raft_consensus.cc:740] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:39.354993 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:39.352212 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.356067 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.356575 31136 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:39.356803 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.358120 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.358741 30983 raft_consensus.cc:3060] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:39.360074 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.363023 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:39.363241 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.364516 31142 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:39.343501  5033 raft_consensus.cc:740] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:39.365180 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.365751 30984 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:39.365479  5033 consensus_queue.cc:260] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.363023 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:39.367233  5379 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.369443  5379 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:39.343072  5596 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.370143 31137 raft_consensus.cc:2410] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:39.370479 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.370390  5596 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.371210 30987 raft_consensus.cc:2393] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:39.371433  5640 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:39.371982 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.372685 30986 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 19: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:39.372946 31136 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:39.373700  5590 consensus_queue.cc:260] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.375054  5590 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.376163  5528 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.378177 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:39.378777 31063 raft_consensus.cc:3060] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:39.372573  5596 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.380065  5688 raft_consensus.cc:3060] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:39.382563 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:39.383775 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:39.384491 31061 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:39.328989  5553 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.344841  5638 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.283703  5669 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.388459 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:39.388935 30991 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:39.374625  5662 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.394773 30984 raft_consensus.cc:2468] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:39.386540  5556 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:39.348589  5095 raft_consensus.cc:2804] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:39.418247 31063 raft_consensus.cc:2468] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
W20260709 12:26:39.420331 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 79217.9us
I20260709 12:26:39.421139 30988 raft_consensus.cc:1240] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:39.424073 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:39.428020 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:39.431586 31694 raft_consensus.cc:3060] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:39.432041 30983 raft_consensus.cc:2468] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:39.432343 30958 leader_election.cc:400] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21.
I20260709 12:26:39.425637 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.432946 30958 leader_election.cc:403] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:39.430315 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:39.433625 31059 raft_consensus.cc:606] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:39.434058 31697 raft_consensus.cc:2393] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:39.431952 31106 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:39.434717 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:39.360050  5572 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:39.435467 31690 raft_consensus.cc:2393] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:39.435804 31060 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:39.434056 31059 raft_consensus.cc:614] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:39.441679 31691 raft_consensus.cc:1240] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:26:39.440232 31664 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.447649  5572 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.419516  5650 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:39.453553 31664 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.454530 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:39.454778  5692 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:26:39.455821 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.456136  5095 raft_consensus.cc:697] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:39.456455 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:39.457010 31692 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.457067  5095 consensus_queue.cc:237] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.458680 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:39.459827  5650 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.442399  5638 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:39.462361  5662 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:39.333911  5667 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:39.473467 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 178899us
W20260709 12:26:39.499543 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 53812.8us
I20260709 12:26:39.501168 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.501024 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.442485  5661 raft_consensus.cc:2804] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:26:39.527010 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.514618 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:39.509858 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.529961 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.530133 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.537498 30983 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:39.538093 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 215203us
I20260709 12:26:39.535000 30992 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:39.541311 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:39.527920 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.541371 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.452051  5431 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21.
I20260709 12:26:39.545579 30983 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:39.549599 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.546499 30991 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:39.571188 30991 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:39.563184 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.549237 31033 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:39.563372 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:39.574065  5521 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:39.549268 31692 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:39.543097 30986 raft_consensus.cc:2376] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24. Re-sending same reply.
I20260709 12:26:39.494410  5601 raft_consensus.cc:2804] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:39.494275  5651 raft_consensus.cc:2804] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:39.565567 31693 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:39.580513 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.582527 30987 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:39.602974  5713 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:39.534417  5378 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.565990  5699 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:39.509272  5647 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:39.509176  5638 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:39.533740  5700 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:39.628600 31143 pending_rounds.cc:85] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 16
I20260709 12:26:39.629020 31143 pending_rounds.cc:107] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 12:26:39.629277 31143 raft_consensus.cc:2889] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:39.611564  5661 raft_consensus.cc:697] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:39.618867 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:39.635854 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:39.562872  5482 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.525943  5379 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.093s
W20260709 12:26:39.642851 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.648254 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.626748  5674 consensus_meta.cc:287] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.002s
W20260709 12:26:39.655994 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.657104 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.614173  5601 raft_consensus.cc:697] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:39.492369  5399 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.617118  5651 raft_consensus.cc:697] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:39.628134  5596 raft_consensus.cc:3055] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:26:39.637303  5671 log.cc:927] Time spent T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.776s	user 0.003s	sys 0.000s
I20260709 12:26:39.668503 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.668057 31661 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:39.564146 30881 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.679239 31139 raft_consensus.cc:1240] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:26:39.659982  5521 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:39.683104  5667 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:39.673508  5665 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:39.689077  5704 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:39.689749  5601 consensus_queue.cc:237] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.612303  5608 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.690904  5661 consensus_queue.cc:237] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.646205  5636 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.701704 31698 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:39.680171 31664 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.709363  5674 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:39.720716 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 183121us
I20260709 12:26:39.652258  5482 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:39.733659  5561 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.003s
I20260709 12:26:39.661335  5700 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:39.730682 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:39.749161 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.728681 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:39.725581  5723 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:39.692821  5596 raft_consensus.cc:740] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:39.660439  5684 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
W20260709 12:26:39.739291 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59230.5us
W20260709 12:26:39.708461 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 107242us
I20260709 12:26:39.755096 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.711953  5650 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:39.733886  5703 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:26:39.778311  5651 consensus_queue.cc:237] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.744987  5572 raft_consensus.cc:2804] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:39.729364  5630 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:39.650799  5640 consensus_queue.cc:1059] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
W20260709 12:26:39.751416 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.757803  5695 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.758517 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.778761  5399 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:39.750519 31060 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:39.783079 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.742553  5553 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.785485 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.788645 30989 raft_consensus.cc:2410] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 12:26:39.759081  5666 log.cc:927] Time spent T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.937s	user 0.005s	sys 0.005s
I20260709 12:26:39.792918 31694 raft_consensus.cc:2468] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:39.794466  5724 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:39.795034  5521 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.798404 31689 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:39.801090 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.803009 31060 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:39.804503 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.801999 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.806522 30990 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:39.806432 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.751739  5561 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.807440 30987 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:39.808264 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.809921 30992 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:39.810448 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.812579 31694 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:39.812811 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.813469 30984 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:39.813920  5725 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:39.814358 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:39.814627  5725 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:39.815065 30984 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:39.802819  5325 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:39.816644 30986 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:39.789968  5496 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:39.818467 30881 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 10 to 13, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 13 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.819752  5712 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:39.798184  5492 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:39.790868  5572 raft_consensus.cc:697] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:39.827515 31698 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:39.786374  5650 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:39.790683  5695 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.839504  5714 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:26:39.840826  5580 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:39.840224 30881 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.820143  5399 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:39.849418  5580 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:39.853366  5580 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.854476  5580 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:39.854293 30881 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.856293 30881 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.858379  5399 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:39.858749  5109 consensus_meta.cc:287] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.002s
I20260709 12:26:39.859149  5109 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.859304  5399 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.860242 30985 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:39.863687 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:39.865325 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 79904.6us
I20260709 12:26:39.866024 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:39.870357 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:39.871407  5580 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:39.871924  5580 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:39.873131  5580 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:39.877606  5733 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:39.847646  5553 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:39.881577 31109 leader_election.cc:400] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:26:39.881966 31109 leader_election.cc:403] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:39.816622  5700 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.857965 30881 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 11 to 14, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 14 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.886294  5482 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.887598 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:39.821704  5699 consensus_queue.cc:1059] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:26:39.866775  5109 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.889751 31109 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:39.890247  5723 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:39.892037  5561 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:39.890151  5572 consensus_queue.cc:237] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.885875 30881 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 8 to 12, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 12 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.896288 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.888113  5650 raft_consensus.cc:2764] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:26:39.888332  5496 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17.
I20260709 12:26:39.926808  5737 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:39.927505  5712 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:39.927699  5737 consensus_queue.cc:1243] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 24.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:39.927148  5636 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:39.931154 31698 raft_consensus.cc:1275] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:39.900166  5723 raft_consensus.cc:2804] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:39.897702  5553 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:39.920166  5561 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:26:39.938834 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57680.6us
I20260709 12:26:39.895660  5714 consensus_queue.cc:1059] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:26:39.920073  5630 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:39.953681 31693 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:39.955044 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:39.957571 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.960168 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:39.958559 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:39.958626 30958 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:39.963049 31059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:39.963627 31059 raft_consensus.cc:606] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:39.964164 31059 raft_consensus.cc:614] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:39.952739  5561 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:39.973305 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:39.973942  5736 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:26:39.971562  5553 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.959914  5723 raft_consensus.cc:697] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 24 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:39.982838  5695 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:39.982936  5723 consensus_queue.cc:237] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:39.984081  5731 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:26:39.959688  5700 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.960167  5630 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:39.989075 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2804) took 1289 ms. Trace:
I20260709 12:26:39.988938  5630 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.989472 31141 rpcz_store.cc:276] 0709 12:26:38.700009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.705310 (+  5301us) service_pool.cc:224] Handling call
0709 12:26:39.988918 (+1283608us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400063e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:39.989018 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726793984}
I20260709 12:26:39.992406 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:39.993785 31143 raft_consensus.cc:1240] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:39.993884 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.994473 31135 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.962393  5687 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.997241 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.997789 31143 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:39.983531  5695 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:39.999235 31137 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:40.000216 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.999034  5714 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:40.002249  5714 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.002671  5714 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.002606  5687 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:39.986644  5584 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:26:39.958930  5712 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:39.991804  5702 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:40.016817 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:39.971155  5734 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:40.029527 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:40.030690 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:40.034085 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:40.038302 31143 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:40.048215 31139 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 24 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:40.049041 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:40.049729 31143 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:40.019845  5734 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:40.045271  5699 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:40.053177 31697 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:40.054164  5730 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:40.030871  5702 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:40.058956 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:40.070093 31135 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
W20260709 12:26:40.073714 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.088385 31063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:40.088949 31063 raft_consensus.cc:606] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:40.089983 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:40.099992 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:40.100703 31067 raft_consensus.cc:2393] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:40.102955 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:40.103699 31068 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:40.105456 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:40.106204 31068 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:40.112213  5741 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:40.112941  5741 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.115486  5741 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.064770  5561 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.127674 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:40.070346  5737 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:26:40.132143  5702 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.095290  5712 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:40.134389 31063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:40.134941 31063 raft_consensus.cc:606] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:40.058182  5553 consensus_queue.cc:1059] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 12:26:40.120532  5699 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:40.149736 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:40.151328 31061 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:40.153972  5399 raft_consensus.cc:3055] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:40.154525  5399 raft_consensus.cc:740] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:40.155649  5399 consensus_queue.cc:260] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.156988  5399 raft_consensus.cc:3060] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:40.158658  5665 raft_consensus.cc:3055] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:40.159210  5665 raft_consensus.cc:740] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:40.170584  5699 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:40.171497  5699 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.175582  5378 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.176100  5378 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.178288  5378 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.172547  5714 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.184048 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2819) took 1044 ms. Trace:
I20260709 12:26:40.184785  5714 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.186021  5521 raft_consensus.cc:3055] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:40.186463  5521 raft_consensus.cc:740] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:40.186836  5677 consensus_queue.cc:1048] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:40.186980  5665 consensus_queue.cc:260] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.187301  5521 consensus_queue.cc:260] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.188237  5665 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:40.188642  5521 raft_consensus.cc:3060] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:40.189996 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.195622 31139 raft_consensus.cc:1240] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:40.196486 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:40.197027 30988 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.197468 30989 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:40.198988  5583 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:40.199678  5553 consensus_queue.cc:1059] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:40.200631  5553 raft_consensus.cc:3055] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:40.200961  5553 raft_consensus.cc:740] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:40.201228  5687 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.190662 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:40.191650  5152 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.192028  5684 raft_consensus.cc:3055] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:26:40.206211 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.208272 31060 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:40.211119 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:40.184432 31144 rpcz_store.cc:276] 0709 12:26:39.139675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.140444 (+   769us) service_pool.cc:224] Handling call
0709 12:26:39.140748 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.178751 (+ 38003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.178798 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.178855 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:39.178911 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.183836 (+1004925us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b540001a754. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:40.183880 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.183994 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86832000}
I20260709 12:26:40.128624 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:40.219767 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:40.223708 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:40.225963 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:40.232123 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:40.226830 31691 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:40.234401 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:40.201748  5553 consensus_queue.cc:260] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.245474 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:40.246358 30989 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:40.244977 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:40.247640 31664 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:40.248014 31697 raft_consensus.cc:1240] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:40.207751  5684 raft_consensus.cc:740] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:40.221103  5583 consensus_queue.cc:1243] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:40.251698 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:40.230306  5547 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:40.230487  5724 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.231964  5659 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:40.256705 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 128142us
W20260709 12:26:40.265120 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.214354  5580 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.252637  5740 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:40.236308  5677 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:40.241575  5676 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:26:40.249183  5723 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:40.322101  5106 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:40.207334  5152 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.326460  5549 raft_consensus.cc:3055] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:40.326954  5549 raft_consensus.cc:740] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:40.327559 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.327945  5152 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.329005  5608 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:40.329891  5740 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.332664  5740 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.341267  5659 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.343495 31690 raft_consensus.cc:1217] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Deduplicated request from leader. Original: 19.15->[21.16-23.17]   Dedup: 21.16->[23.17-23.17]
I20260709 12:26:40.343827  5659 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.236272  5556 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:26:40.345578 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:40.347322 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:40.252768  5378 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:40.255900  5539 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:40.269687  5594 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:40.294914 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.301316 31144 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:26:40.301358  5684 consensus_queue.cc:260] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.305630  5724 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.296108  5630 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:40.315764 31135 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 24 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:40.316534  5421 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:40.317965  5596 consensus_queue.cc:260] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.320713 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 3093) took 1140 ms. Trace:
I20260709 12:26:40.367496 30985 rpcz_store.cc:276] 0709 12:26:39.179836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.179997 (+   161us) service_pool.cc:224] Handling call
0709 12:26:40.320415 (+1140418us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54000762d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:40.320623 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400171776}
I20260709 12:26:40.344019 31690 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: [21.16-23.17]
I20260709 12:26:40.272812 30989 raft_consensus.cc:1240] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:26:40.351663 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:40.357833 31694 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:40.358765 30988 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
W20260709 12:26:40.359526 31033 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.361039 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.370760 30984 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:26:40.372030 30990 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:40.352300  5378 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.354702  5539 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.371377 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:40.376709 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:40.353859  5594 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.354327  5664 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:40.381449 31696 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:40.354511  5521 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.364332  5421 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:40.396791 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:40.373443  5401 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.372807  5710 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:40.376312  5378 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:40.327950  5549 consensus_queue.cc:260] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.418292 30985 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:26:40.416218 31060 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:26:40.385600  5539 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:40.440622 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:40.423656 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.390833  5596 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:40.454172 30991 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:40.450439 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:40.387604  5594 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.397612  5729 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:40.430603 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:40.462443 31060 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:40.464713 30983 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:40.401386  5590 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:40.390618  5496 consensus_queue.cc:1059] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:26:40.398981  5730 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:40.448076  5676 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.412026  5580 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.447974  5659 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.476481 30990 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:40.418772  5664 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:40.389006  5521 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.451951  5556 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.481467 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1462) took 1342 ms. Trace:
I20260709 12:26:40.399852  5714 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.077s
I20260709 12:26:40.418653  5401 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.447721  5325 consensus_queue.cc:1059] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:26:40.482012 31059 raft_consensus.cc:1275] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:40.466027  5379 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.492974 31689 rpcz_store.cc:276] 0709 12:26:39.139123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.141929 (+  2806us) service_pool.cc:224] Handling call
0709 12:26:39.142191 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.798754 (+656563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.481251 (+682497us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b540076ded4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:40.481415 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508330752}
I20260709 12:26:40.399260  5699 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:40.452131  5583 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:26:40.449769  5710 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.474644  5590 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:40.521641 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 71318.1us
I20260709 12:26:40.523023 30990 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:40.524176 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:40.527647 30992 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:40.476114  5661 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:40.532989 30988 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:40.482728  5730 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.534878 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:40.535800 31139 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:40.536134  5699 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:40.536522  5695 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:40.527264  5325 consensus_queue.cc:1243] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:40.539912 30983 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2819) took 1009 ms. Trace:
W20260709 12:26:40.543263  5754 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:40.544697  5033 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:40.545015 30986 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:40.517266  5556 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.510313  5399 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:40.562146 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:40.588631 31143 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:40.564217 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:40.524646  5590 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:40.579790 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:40.539350  5521 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.577821  5580 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:40.581701 30983 rpcz_store.cc:276] 0709 12:26:39.530952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.550188 (+ 19236us) service_pool.cc:224] Handling call
0709 12:26:40.539759 (+989571us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b540014e6d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:40.539859 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2008099200}
I20260709 12:26:40.488801  5676 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.508849  5659 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.554859  5553 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:40.549045  5723 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:40.559065  5547 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.557453  5737 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:40.508662  5664 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.488143  5421 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.569819  5608 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.521121  5379 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.593309 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:40.524196  5561 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 12:26:40.563985  5572 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:26:40.621392 31061 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:40.621970 31061 raft_consensus.cc:606] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:40.566010  5517 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:26:40.479599  5729 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.603446  5714 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:40.578691  5492 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:26:40.623615 31143 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:40.629194 31068 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:40.603251  5724 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.578670  5661 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.605155  5733 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:40.629194  5421 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:40.608778  5547 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.634519  5561 consensus_queue.cc:1243] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 24.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:40.615115  5677 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:26:40.631842  5760 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:40.663749 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.664184  5695 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.670240  5636 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:40.610966  5737 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:40.630579  5696 consensus_queue.cc:1175] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" responder_term: 20 status { last_received { term: 20 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 20 index: 11 } }
I20260709 12:26:40.625649  5608 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:40.682600 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.693622 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:40.687417 30991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
W20260709 12:26:40.696192 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.680420  5492 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:40.656255  5724 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:40.655793  5583 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:40.758945 31058 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:26:40.682693  5760 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.772981 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:40.655623  5714 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:26:40.755867  5721 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:40.684343  5636 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:40.773847 30992 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:40.773600 31060 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:40.692775  5762 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:40.791895 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:40.820050 30989 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:26:40.824015 30989 pending_rounds.cc:85] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 13
I20260709 12:26:40.824460 30989 pending_rounds.cc:107] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:26:40.824910 30989 raft_consensus.cc:2889] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:40.827646 31693 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:40.799467 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:40.824366 30955 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:40.830281 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:40.757716  5109 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:40.835976 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.646409  5699 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.831240 30986 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 24.
I20260709 12:26:40.845229 30988 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:26:40.849923 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45558.6us
I20260709 12:26:40.694244  5695 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.698295  5492 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.790477  5723 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.777241  5763 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:40.775732  5760 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:40.780725  5547 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:40.777992  5556 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.778445  5700 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.121s
I20260709 12:26:40.849305  5757 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:40.947149 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 89221.8us
I20260709 12:26:40.992182  5757 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.993485 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:40.775393  5583 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.994766  5768 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:40.996769  5583 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.949640  5677 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.998661  5677 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.999614 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:41.000485 31067 raft_consensus.cc:2410] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:26:41.001251  5677 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:40.950373  5758 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 12:26:40.950502  5765 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:41.003394 31065 raft_consensus.cc:1275] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:40.951362  5729 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:40.954873  5761 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:26:41.006261  5769 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:41.010268  5651 consensus_queue.cc:1048] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:26:40.957931  5665 consensus_meta.cc:287] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.000s
I20260709 12:26:40.959563  5695 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.960645  5661 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.961539  5401 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.967056  5737 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.967592  5608 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.971288  5659 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.836977  5762 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.974462  5730 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.936347  5475 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
W20260709 12:26:40.977165  5378 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.000s
I20260709 12:26:40.823017  5721 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.773553  5724 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.857589  5727 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:41.045567 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:40.979013  5596 consensus_meta.cc:287] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.519s	user 0.006s	sys 0.002s
W20260709 12:26:40.979112  5106 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.000s
I20260709 12:26:40.851985  5109 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.050810 31137 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:41.052188  5109 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:40.983741  5700 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.058694 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.059324 31144 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:40.942731  5723 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:40.984476  5636 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:40.986991  5664 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.947889  5594 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.072237  5594 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:40.990856  5763 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.074267  5763 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.076290  5724 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.079830 31138 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:41.012406  5765 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.023350  5757 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.041483  5677 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.020303  5687 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.969924  5590 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.048257 31136 raft_consensus.cc:1275] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:41.022980  5492 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:41.040274  5665 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.062778 31141 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:26:41.069536  5760 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:41.047349  5539 consensus_meta.cc:287] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.000s
I20260709 12:26:41.014708 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:41.048059  5727 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.087193 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.130575 31144 raft_consensus.cc:2410] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:26:41.052783  5496 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.095119 31136 raft_consensus.cc:1240] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:26:41.113188 31059 raft_consensus.cc:2393] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:41.113963  5760 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.114991  5721 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.118485  5710 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.048416  5378 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.125941 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45360 (request call id 2934) took 1092 ms. Trace:
I20260709 12:26:41.174029 31137 rpcz_store.cc:276] 0709 12:26:40.033642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.033806 (+   164us) service_pool.cc:224] Handling call
0709 12:26:41.125728 (+1091922us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540029e0d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:41.125860 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2335930112}
I20260709 12:26:41.129819 31060 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:41.176106 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:41.176331 30955 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:41.176662 31060 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.081418  5700 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.189018 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:41.192935 31144 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:26:41.181878  5033 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.001s
I20260709 12:26:41.170843  5727 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:41.088024  5677 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.182746  5421 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:41.095947  5665 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.203279 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:41.092995  5687 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:41.204092 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.214386 31140 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:26:41.178921  5539 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.178630  5496 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.202278  5707 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.205457  5421 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.220425 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:41.212594  5399 consensus_meta.cc:287] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 12:26:41.209035  5760 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:41.257233 30991 raft_consensus.cc:606] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:41.257728 30991 raft_consensus.cc:614] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:41.245000 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:41.283669 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.256047  5421 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.294536  5095 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.305555  5095 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.307289  5665 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:41.308099  5095 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.308277 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:41.309284 31067 raft_consensus.cc:2393] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 15: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:41.297446 31033 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 2.100s (SENT) [suppressed 6 similar messages]
I20260709 12:26:41.316190 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:41.259665  5760 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:41.251322 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.250886 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.259562  5399 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:41.255246  5707 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:41.411780 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.474851 31144 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:41.332835 31060 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
W20260709 12:26:41.471199  5553 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.863s	user 0.007s	sys 0.003s
I20260709 12:26:41.503156 31143 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:26:41.352108  5760 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:41.351364  5669 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.513244 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:41.513361  5669 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.357805  5762 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:41.487242 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:41.468034  5539 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.514813 31061 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:41.540591 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:41.486629  5776 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.543556 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:41.469580  5421 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.554481 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:41.557076 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:41.559118 30987 raft_consensus.cc:1240] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:41.549301  5776 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.565735 30984 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 3115) took 1334 ms. Trace:
I20260709 12:26:41.571126 30984 rpcz_store.cc:276] 0709 12:26:40.231635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.243609 (+ 11974us) service_pool.cc:224] Handling call
0709 12:26:40.370551 (+126942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.371938 (+  1387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.565546 (+1193608us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540022d8d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:41.565677 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291241344}
W20260709 12:26:41.571321  5753 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.061s	user 0.003s	sys 0.003s
I20260709 12:26:41.560292  5762 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:41.574260  5700 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.564720  5776 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:41.578509  5772 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.005s	sys 0.009s
I20260709 12:26:41.583657  5677 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.569063  5496 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:41.613386  5728 raft_consensus.cc:3055] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:41.613821  5728 raft_consensus.cc:740] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:41.594573  5762 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.618352 31690 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:26:41.623639 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 326204us
I20260709 12:26:41.593753  5492 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
W20260709 12:26:41.628898 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.612536  5772 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.006s	sys 0.010s
I20260709 12:26:41.597690  5775 consensus_queue.cc:1059] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:26:41.630831 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.634017 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:41.634624 31694 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:41.640422 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.564402  5734 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.654923  5734 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:41.656003  5539 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.656735  5775 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:41.656514  5539 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.657229  5775 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:41.658690  5781 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:26:41.607648  5687 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.661424  5549 raft_consensus.cc:3060] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:41.665798  5687 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.666447  5687 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.671468  5669 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.672015  5721 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.672438  5721 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:41.674154  5790 log.cc:927] Time spent T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:26:41.674983  5784 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
I20260709 12:26:41.677289 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.623793  5762 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:41.684839 31694 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:41.683378  5684 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.685328  5669 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.690109  5378 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:41.711249  5721 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.696393  5421 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.692770  5539 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.732182 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1466) took 1715 ms. Trace:
I20260709 12:26:41.704492  5669 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.715706  5723 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.717032  5687 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.732517 31698 rpcz_store.cc:276] 0709 12:26:40.016689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.017244 (+   555us) service_pool.cc:224] Handling call
0709 12:26:40.019218 (+  1974us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:40.026122 (+  6904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.026166 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:40.040584 (+ 14418us) log.cc:844] Serialized 78 byte log entry
0709 12:26:40.358064 (+317480us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:40.373088 (+ 15024us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.382340 (+  9252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.382444 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.541989 (+159545us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b18004e46c0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:41.729717 (+1187728us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b18004e46c0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:41.731941 (+  2224us) raft_consensus.cc:1685] finished
0709 12:26:41.731962 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.732120 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14877,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":2536,"raft.queue_time_us":189,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":10032,"spinlock_wait_cycles":20001408,"thread_start_us":287744,"threads_started":1,"wal-append.queue_time_us":346249}
I20260709 12:26:41.714092  5575 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:26:41.757355  5669 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:41.766933 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.737383  5760 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.779124  5601 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.779913  5760 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:41.780473  5786 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 12:26:41.779666  5601 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.782814 31142 raft_consensus.cc:1275] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:41.783046 31692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 1436) took 1593 ms. Trace:
I20260709 12:26:41.727622  5775 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.783459 31692 rpcz_store.cc:276] 0709 12:26:40.189353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.189566 (+   213us) service_pool.cc:224] Handling call
0709 12:26:40.189900 (+   334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:40.190247 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.190271 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:40.783854 (+593583us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b2c000b6068. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:26:40.784496 (+   642us) log.cc:844] Serialized 52 byte log entry
0709 12:26:41.137073 (+352577us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:41.187424 (+ 50351us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.187804 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.187978 (+   174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.188067 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.755979 (+567912us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b18004c4860. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:41.780697 (+ 24718us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b18004c4860. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:41.782821 (+  2124us) raft_consensus.cc:1685] finished
0709 12:26:41.782848 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.782979 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":72651,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":5498,"raft.queue_time_us":16986,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":6145,"spinlock_wait_cycles":88368000,"thread_start_us":352166,"threads_started":1,"wal-append.queue_time_us":424012}
I20260709 12:26:41.787268 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:41.788195  5778 log.cc:927] Time spent T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:26:41.746006  5379 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.767379  5777 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.769623  5723 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.804884 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.795297 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.789050  5601 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:41.809402 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 216525us
W20260709 12:26:41.813160  5727 consensus_meta.cc:287] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.005s
I20260709 12:26:41.813555  5727 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.814975 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:41.816960 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:41.821069 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:41.833010 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:41.834671 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:41.837886 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.822881  5389 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.841678  5601 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.842350  5601 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.844931  5601 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.846937  5399 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:41.850205  5727 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:41.838321  5496 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:41.820266  5775 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:41.835798  5665 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:41.852928  5760 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:26:41.811877  5723 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.858455 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.866978 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.822842  5733 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:41.835196  5492 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:41.859995  5496 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:41.861411 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 171620us
I20260709 12:26:41.861513 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.862854  5684 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:41.866758 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.810670  5777 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.867996 31136 raft_consensus.cc:2393] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:41.868840 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.872270 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.878789 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.879964 31135 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:41.880030 31137 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:41.855877  5775 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.883520 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:41.889715 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.890436 31139 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:41.889698 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.891888 31137 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 25.
W20260709 12:26:41.893972  5783 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.401s	user 0.000s	sys 0.001s
I20260709 12:26:41.895203  5700 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:41.895742  5700 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:41.893843  5676 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.896647  5700 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.898463 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 3124) took 1205 ms. Trace:
I20260709 12:26:41.898860 31061 rpcz_store.cc:276] 0709 12:26:40.693240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.693358 (+   118us) service_pool.cc:224] Handling call
0709 12:26:41.898295 (+1204937us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b540027c254. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:41.898401 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887117696}
I20260709 12:26:41.901260 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.901896 31694 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:41.887846 31136 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:41.909933 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.910156 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:41.910895 31136 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.912067 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.910784 31698 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:41.929369  5399 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:41.929940  5399 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.931337  5728 consensus_queue.cc:260] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.931988  5399 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.932744  5728 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:41.886879  5775 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:41.940593 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:41.941309 31137 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.935571  5760 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:41.943222  5760 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.945333  5760 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.955835  5727 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:41.956398  5727 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.958511  5727 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.960240  5777 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.963488  5723 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:41.875205  5733 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:41.973865 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.943235 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.981297 31139 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:41.976066 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.982447 31135 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:41.978199 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:41.979036  5723 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.878054  5496 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:41.991479 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:41.997812 31135 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:41.992164 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.001797 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:42.002810 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:42.003383 31135 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:42.003902 31136 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:41.985314  5723 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.009832  5723 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:42.011766 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.012598  5760 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.012770  5765 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.013002  5399 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.013108  5760 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.013478  5399 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.014180 30991 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55468 (request call id 6) took 1337 ms. Trace:
I20260709 12:26:42.014549 30991 rpcz_store.cc:276] 0709 12:26:40.687074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.687244 (+   170us) service_pool.cc:224] Handling call
0709 12:26:42.013963 (+1326719us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b5400312c54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88e853 00007f2ef085c60a 00007f2ee9de27d4 00007f2ee9de2762 00007f2ee9de2691 00007f2ee9de260c 00007f2ee9de18c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:42.014085 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1293117312}
I20260709 12:26:42.015369  5760 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.015545  5399 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:42.012786 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.016866  5553 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:42.017585 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.018651  5760 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:42.019227 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:42.020359  5788 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.484s	user 0.000s	sys 0.001s
I20260709 12:26:42.019671  5760 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.021608 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.022693 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:42.022746 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.023281 31059 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.018812 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.023438  5760 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.024189 31061 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:42.024542 31062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1829) took 1027 ms. Trace:
I20260709 12:26:42.025007 31062 rpcz_store.cc:276] 0709 12:26:40.997594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.002377 (+  4783us) service_pool.cc:224] Handling call
0709 12:26:41.002629 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.489743 (+487114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.489781 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.489826 (+    45us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:41.489863 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.024303 (+534440us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54000c5b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:42.024348 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.024483 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8231,"raft.queue_time_us":4833,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":11959,"spinlock_wait_cycles":1109326720}
I20260709 12:26:42.017352 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.021622 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1343) took 1223 ms. Trace:
I20260709 12:26:41.889153  5684 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.017352  5553 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.035866 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.043322 31139 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:42.041349 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.042515 30985 rpcz_store.cc:276] 0709 12:26:40.798467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.798982 (+   515us) service_pool.cc:224] Handling call
0709 12:26:40.799291 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.053300 (+254009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.053344 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.053800 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.222496 (+168696us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:41.222550 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.779970 (+557420us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:42.021322 (+241352us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54002c1d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:42.021376 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.021427 (+    51us) raft_consensus.cc:1685] finished
0709 12:26:42.021439 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.021567 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6304,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":19091,"spinlock_wait_cycles":686333568,"thread_start_us":168382,"threads_started":1,"wal-append.queue_time_us":417552}
W20260709 12:26:42.046087 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44406.1us
I20260709 12:26:42.046264 31143 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:42.027582 31058 raft_consensus.cc:2410] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:26:42.018651 31065 raft_consensus.cc:1275] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:42.027606 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.051394 31060 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:42.052541 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.059791  5601 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:42.059981  5561 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.060390  5601 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.060932  5601 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.060705  5561 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.062384  5378 consensus_queue.cc:1048] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:42.074208 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:42.076135 31068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51946 (request call id 3123) took 1636 ms (client timeout 1599 ms). Trace:
W20260709 12:26:42.078058 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 51099.8us
I20260709 12:26:42.079965 31136 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:26:42.081851 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:42.076606 31068 rpcz_store.cc:269] 0709 12:26:40.440141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.440281 (+   140us) service_pool.cc:224] Handling call
0709 12:26:42.075967 (+1635686us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002cf1d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:42.076064 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415927168}
I20260709 12:26:42.085565 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.086503 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.088901 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.089078 31065 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:42.089165 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.089664 30991 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:42.090168 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:42.091806 30992 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49530 (request call id 2831) took 1531 ms (client timeout 1564 ms). Trace:
W20260709 12:26:42.092298 30992 rpcz_store.cc:269] 0709 12:26:40.560837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.560969 (+   132us) service_pool.cc:224] Handling call
0709 12:26:42.091623 (+1530654us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400225954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:42.091744 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423231872}
I20260709 12:26:42.094547 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.034518  5801 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.096767 31140 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:42.096939 31698 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:42.097954 30986 raft_consensus.cc:2393] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:42.098282 30990 raft_consensus.cc:2393] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 24: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:42.081965  5378 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:42.107278 31139 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:42.050853  5728 consensus_queue.cc:1048] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:26:42.109182 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:42.088538  5664 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.014s	sys 0.075s
I20260709 12:26:42.108455 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.119990 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.120595 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.147944 31067 raft_consensus.cc:2393] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:42.107513 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.103919  5496 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:42.138768 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:42.069886  5777 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:26:42.149199 31690 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 21: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:42.154557 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:42.152258 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.147663 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.149961 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.143455 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.151214 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:42.083400  5572 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:26:42.130283  5325 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.166190 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.182372 31139 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.182497 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.181181 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:42.186002 30990 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:42.187422 31694 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
W20260709 12:26:42.146256  5664 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.014s	sys 0.075s
I20260709 12:26:42.151141  5389 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.205018 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.205621 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.206350 30990 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 16: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:42.205636 31140 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:42.207783 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.208690 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:42.208770 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.209522 30986 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 22 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:26:42.209750 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.210140 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.210542 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.210753 30989 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:42.211568 30984 raft_consensus.cc:2410] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:26:42.215368  5684 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.215960 31140 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:42.217279 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.217653  5389 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.218189  5665 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.218586  5665 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.218912  5665 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.219810  5389 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.221140 31140 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:42.222291 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:42.222855 30985 raft_consensus.cc:3060] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.224366 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.225075 30984 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:42.226573 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.234735 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.235617 31058 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:42.236765  5496 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:42.237160  5496 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:42.243508 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:42.250778 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.146760  5801 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.237560  5561 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.258715  5760 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:26:42.261005  5669 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:42.261857  5669 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.262575  5801 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:42.263191  5787 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.636s	user 0.003s	sys 0.000s
I20260709 12:26:42.264117  5669 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.266171  5496 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.267536  5496 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:42.262584  5760 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:42.268090  5760 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:42.268508  5760 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:42.264915  5782 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.751s	user 0.001s	sys 0.002s
I20260709 12:26:42.225759  5805 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.270608  5805 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.271513 31063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1833) took 1258 ms. Trace:
I20260709 12:26:42.272881  5601 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.265601 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1345) took 1251 ms. Trace:
I20260709 12:26:42.272080 31063 rpcz_store.cc:276] 0709 12:26:41.013307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.013895 (+   588us) service_pool.cc:224] Handling call
0709 12:26:41.014191 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.014691 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.014715 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.015144 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.462871 (+447727us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:41.462934 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.463147 (+   213us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.237750 (+774603us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:42.271171 (+ 33421us) raft_consensus.cc:1685] finished
0709 12:26:42.271200 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.271446 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":820,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2597,"spinlock_wait_cycles":2588544,"thread_start_us":447361,"threads_started":1,"wal-append.queue_time_us":495402}
I20260709 12:26:42.273326  5805 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.273751 30988 rpcz_store.cc:276] 0709 12:26:41.013671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.013805 (+   134us) service_pool.cc:224] Handling call
0709 12:26:41.026055 (+ 12250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.026372 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.026402 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.032043 (+  5641us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.210853 (+178810us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:41.210892 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.210958 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.262645 (+1051687us) spinlock_profiling.cc:243] Waited 52.7 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:42.265229 (+  2584us) raft_consensus.cc:1685] finished
0709 12:26:42.265259 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.265530 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1883,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3018,"spinlock_wait_cycles":121116032,"thread_start_us":135928,"threads_started":1,"wal-append.queue_time_us":567463}
I20260709 12:26:42.265120  5665 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.150199  5762 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.285619 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.235188  5707 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:42.279986  5734 consensus_meta.cc:287] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.003s
I20260709 12:26:42.289944  5590 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.291111 31064 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:26:42.291316  5733 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.258548  5684 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.258836  5389 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.201390  5325 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.293876 31692 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:42.294677 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.296386 31063 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:42.294996 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:42.297988 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 210182us
I20260709 12:26:42.299171 31140 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:42.301440 31137 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:42.436108 31135 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:42.438504 30985 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:42.343521  5553 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.309046 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:42.462111 31063 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:42.310043 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.311303 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:42.463423 31068 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:42.259564  5399 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:42.292744  5549 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.465420  5549 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.322184 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:42.467875 31067 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:42.324710 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.470570 31065 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:42.471524 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.472537 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.473047 31065 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:42.473740 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.325527  5707 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.474205  5492 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.474480  5707 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.474915  5492 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.476841  5707 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.477694  5492 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:42.326198 30955 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:42.327557  5665 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:42.328927 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:42.482802 30988 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.331634 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1828) took 1334 ms. Trace:
I20260709 12:26:42.486402 31064 rpcz_store.cc:276] 0709 12:26:40.997318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.001950 (+  4632us) service_pool.cc:224] Handling call
0709 12:26:41.002194 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.291571 (+1289377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.331413 (+ 39842us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400383e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:42.331574 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2963169024}
I20260709 12:26:42.487735 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.488641 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.489137 31064 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:42.489889 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:42.128618 31059 raft_consensus.cc:2468] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:42.339572  5762 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:42.295900  5684 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.341008 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:42.295459  5389 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:42.343950 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 265094us
W20260709 12:26:42.503924 30958 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.045s	sys 0.040s
I20260709 12:26:42.504026 31064 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:42.504648 30958 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.045s	sys 0.040s
I20260709 12:26:42.344688 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.346057 30992 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:26:42.505477 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 597309us
I20260709 12:26:42.347033 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.506501 31062 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:42.370000 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:42.423441 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.427116  5033 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.430675 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1355) took 1383 ms. Trace:
I20260709 12:26:42.510432 31142 rpcz_store.cc:276] 0709 12:26:41.047163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.047267 (+   104us) service_pool.cc:224] Handling call
0709 12:26:41.049743 (+  2476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.783203 (+733460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.430342 (+647139us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54001eaad4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:42.430595 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1685176704}
I20260709 12:26:42.511096 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:42.511248 30988 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:42.304018 31139 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:42.512383 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.513101 31139 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
W20260709 12:26:42.440812 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.484990 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.335239  5601 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.501762 31061 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:42.503033 31065 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
W20260709 12:26:42.508467 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 187953us
I20260709 12:26:42.524919 30955 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:42.430815 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 87177.6us
I20260709 12:26:42.439888 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.527307 31137 raft_consensus.cc:2393] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:42.528050 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.529276 31063 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:42.514889 31135 raft_consensus.cc:2410] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:42.514098 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.517437 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.523504 31067 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:42.526590 31661 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:42.536063 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.531221 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.536715 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.537350 31142 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:42.537758 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.536754 31691 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 25.
I20260709 12:26:42.538476 30985 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:42.553040 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.553638 31059 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:42.549897 31064 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:42.510309  5389 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.558712  5677 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:26:42.574323 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.541112  5482 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:26:42.562141 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:42.536373  5721 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:42.567612  5684 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.576897 31697 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:42.568864  5325 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.542279  5492 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:42.509701  5687 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:42.586697 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.587291 31142 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:42.588231 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.588723 31142 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:42.589609 31068 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
W20260709 12:26:42.591017 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 347568us
I20260709 12:26:42.555661  5152 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.609606 30985 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:42.620208 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:42.620741 30985 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.610626 31033 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:42.614933 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.609833 30988 raft_consensus.cc:1240] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:42.622897 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:42.623042 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:42.623548 30988 raft_consensus.cc:2376] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18. Re-sending same reply.
I20260709 12:26:42.624226 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.624414 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.624941 30988 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:42.625764 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.610940 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.626895 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.627460 31137 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:42.628194 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.623553 30991 raft_consensus.cc:2368] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 23. Current term is 24.
I20260709 12:26:42.618757 31697 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:42.532164  5033 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:42.636379 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 109864us
W20260709 12:26:42.630890 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 94746us
I20260709 12:26:42.629361 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:42.646934 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.647115 30955 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.648386 30955 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.641927 30985 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:42.599979  5492 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:42.654649 30986 raft_consensus.cc:2368] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20.
I20260709 12:26:42.608405  5721 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:42.668732 30984 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:42.645485  5733 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:42.679005 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.608628  5325 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:26:42.685815 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.688740 30985 raft_consensus.cc:2393] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 21: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:42.656006  5734 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:42.690672 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.691370 30991 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:42.630440  5152 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.694684 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.695983 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:42.688063  5775 consensus_meta.cc:287] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.003s
I20260709 12:26:42.698024 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.698884 30991 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 25.
I20260709 12:26:42.698024 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.704532 30985 raft_consensus.cc:2376] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21. Re-sending same reply.
I20260709 12:26:42.689661  5575 raft_consensus.cc:2764] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:26:42.690251  5734 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.689460  5826 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:42.689710  5733 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:42.697423 30984 raft_consensus.cc:2468] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 25.
W20260709 12:26:42.712595 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.714005 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.711512 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.718922 30985 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:26:42.720613 31143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2836) took 1208 ms (client timeout 1599 ms). Trace:
W20260709 12:26:42.721096 31143 rpcz_store.cc:269] 0709 12:26:41.512053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.861250 (+349197us) service_pool.cc:224] Handling call
0709 12:26:42.720412 (+859162us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540006fed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:42.720541 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391946624}
I20260709 12:26:42.732514 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:42.733590  5492 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.735824 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:42.736584 31065 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.715211  5826 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:42.737305  5826 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.738626 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.739401 31063 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:42.705684  5721 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.740916  5762 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:42.742660  5776 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:42.743656  5810 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:26:42.743340  5776 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:42.744987  5812 log.cc:927] Time spent T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:26:42.689577  5561 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:26:42.741614  5762 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.745882  5561 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:42.746333  5561 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:42.749612  5664 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:42.750108  5664 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.756075 31106 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:42.752012  5677 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:42.696346  5553 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:26:42.760983  5760 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:42.761176 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 123595us
I20260709 12:26:42.762563  5033 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.762769  5482 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.763106  5647 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:42.761087 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 115889us
W20260709 12:26:42.764212  5802 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.615s	user 0.004s	sys 0.004s
I20260709 12:26:42.841432 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.713321  5734 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:42.839175 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:42.846447 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58866.1us
I20260709 12:26:42.847199 31661 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:42.852753 31693 raft_consensus.cc:2393] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 25: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:42.752086  5811 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.707500  5775 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.852489 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:42.778944 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:42.855922 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.835938  5482 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.862506 31144 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:42.861016 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.864606 31058 pending_rounds.cc:85] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 14
I20260709 12:26:42.873732 31058 pending_rounds.cc:107] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:26:42.874154 31058 raft_consensus.cc:2889] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:42.875758 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:42.876171 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 97336.9us
I20260709 12:26:42.840756  5561 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.856644  5601 raft_consensus.cc:2804] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:42.878041  5561 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.878648  5152 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.879933 31689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1477) took 1263 ms. Trace:
I20260709 12:26:42.880389 31689 rpcz_store.cc:276] 0709 12:26:41.616533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.617492 (+   959us) service_pool.cc:224] Handling call
0709 12:26:41.627786 (+ 10294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.628181 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.628225 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.628683 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.875311 (+246628us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:41.875379 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.875503 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.763706 (+888203us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:42.879605 (+115899us) raft_consensus.cc:1685] finished
0709 12:26:42.879635 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.879863 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1058,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":6636928,"thread_start_us":246277,"threads_started":1,"wal-append.queue_time_us":468850}
I20260709 12:26:42.883225  5630 consensus_queue.cc:1048] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:42.883639  5775 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.856158  5811 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:42.886319  5811 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:42.878068  5817 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:26:42.888449 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:42.889325  5630 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:42.814064  5695 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:26:42.845017  5647 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.791783  5389 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.894465  5647 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.895321 31139 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:42.895753  5776 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:42.897480  5695 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:42.898085  5760 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.843791  5700 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:42.875195  5733 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.900236  5762 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.901398  5665 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.901921  5665 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:42.898578 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 59492.6us
I20260709 12:26:42.902340  5665 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.903476 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:42.904716  5700 raft_consensus.cc:2764] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:26:42.894577  5389 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.905403 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:42.906694  5762 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:42.908783  5762 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 16.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:42.909911  5776 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:42.856503  5827 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:26:42.910318  5776 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:42.910583  5827 consensus_queue.cc:1243] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:26:42.911545  5776 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:42.912230  5496 consensus_meta.cc:287] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.002s
I20260709 12:26:42.875982  5664 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:42.914343 31068 raft_consensus.cc:2410] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:26:42.914736 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:42.918144  5657 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.918571 31065 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:42.918618  5657 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.935577 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2825) took 1879 ms. Trace:
I20260709 12:26:42.925398  5664 consensus_queue.cc:1059] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 12:26:42.935583  5700 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
W20260709 12:26:42.939843  5818 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.369s	user 0.000s	sys 0.002s
I20260709 12:26:42.919723  5669 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:42.939525 31141 rpcz_store.cc:276] 0709 12:26:41.055955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.096575 (+ 40620us) service_pool.cc:224] Handling call
0709 12:26:42.022908 (+926333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.023128 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.023156 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.023618 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.471911 (+448293us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:42.471963 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.472047 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.745135 (+273088us) raft_consensus.cc:1685] finished
0709 12:26:42.745163 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.935293 (+190130us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b54001ea0d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:42.935510 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1954767104,"thread_start_us":440007,"threads_started":1,"wal-append.queue_time_us":596168}
I20260709 12:26:42.941205  5657 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:42.920993  5808 log.cc:927] Time spent T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.361s	user 0.000s	sys 0.003s
I20260709 12:26:42.925837 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:42.943128  5561 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:42.944941 31106 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:42.945499 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 68602.2us
I20260709 12:26:42.945777  5496 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:42.946249  5549 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:42.941048  5636 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:42.947748  5776 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:42.948926 31064 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:42.950999 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.951581 31141 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:42.940183  5664 consensus_queue.cc:1243] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 12:26:42.955753  5825 log.cc:927] Time spent T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:26:42.956534 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.958925 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:42.959154  5399 raft_consensus.cc:3055] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:42.959708  5399 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:42.960093 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.960881 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:42.960642  5399 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.920377  5601 raft_consensus.cc:697] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 21 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:42.961778  5399 raft_consensus.cc:3060] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:42.793253  5677 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:42.963409 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:42.920209 31694 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:26:42.928306 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.865607 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:42.980206 30958 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.969270 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:42.980528 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.981038 31065 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:42.973318 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:42.974169 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:42.983203 30990 raft_consensus.cc:3060] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.982054 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.985546  5707 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.985915  5707 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.986261  5707 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.973433 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:42.987828 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:42.989193  5760 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:42.977797 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:42.961896  5561 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:26:42.991375 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 343100us
I20260709 12:26:42.991782 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:42.993435 30990 raft_consensus.cc:2468] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:42.995376 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:43.002794 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:43.004698 31693 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:43.005399 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:43.007946 30958 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:43.008177 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:43.015692 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:43.009073 30958 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:43.017163 30958 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:43.021248 31140 raft_consensus.cc:1275] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:42.990597  5700 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.034354 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:43.039755 31143 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:43.035785 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:43.027446  5033 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:43.036913 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.994575  5695 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:42.994482  5561 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:43.040639 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:43.003046  5389 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:43.043247  5033 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.046526 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:42.990514  5669 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.060s
I20260709 12:26:43.006003  5775 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:43.038622  5575 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:43.051473  5803 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:43.055783  5677 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:43.071743  5734 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.005802  5496 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:43.001798  5482 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:43.041533 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:42.930640  5826 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:42.962109  5601 consensus_queue.cc:237] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.171921 30987 raft_consensus.cc:1240] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 24. Current term is 25. Ops: [22.15-24.16]
I20260709 12:26:43.081638  5695 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.109412  5389 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.110796  5664 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:43.225070  5389 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:43.225224  5664 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:43.108537  5561 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:43.122112  5033 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:43.230597 31136 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:26:43.230990 31136 pending_rounds.cc:85] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 12
I20260709 12:26:43.231207 31136 pending_rounds.cc:107] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:26:43.231436 31136 raft_consensus.cc:2889] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:43.232431  5815 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.232936  5815 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:43.176648  5575 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:43.234845  5575 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:43.202323 31061 raft_consensus.cc:1240] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 24. Current term is 25. Ops: [22.17-24.18]
I20260709 12:26:43.212667  5665 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.222795  5695 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:43.192648  5657 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.224076  5549 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:43.197361  5803 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:43.095239  5811 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.140818  5707 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:43.278644 31692 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:26:43.290179 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 294867us
I20260709 12:26:43.274523  5378 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:43.314823 30992 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.005s	sys 0.020s
I20260709 12:26:43.327142 31106 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:43.325419 30992 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.005s	sys 0.020s
I20260709 12:26:43.265255  5549 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.354357 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:43.358908 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:43.384235 31106 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:43.295375  5684 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:43.387010 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 368878us
I20260709 12:26:43.392262 31106 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:43.390112 30958 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:43.356006  5152 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:43.353750  5733 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:43.397018 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:43.380908  5636 raft_consensus.cc:2764] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:26:43.379110  5677 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:43.396461  5775 raft_consensus.cc:2764] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:26:43.423653 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:43.355189  5378 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:43.408059 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 391650us
I20260709 12:26:43.435304 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:43.438503 30985 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:43.414830  5831 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:43.407580  5528 consensus_queue.cc:1048] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:26:43.439073 31109 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:43.390025  5803 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:43.433925  5721 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:43.430070  5496 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:43.436678  5636 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:43.458851 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:43.457538 31109 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:43.445248  5831 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.413381  5733 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.518105 31139 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:43.507743  5734 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:43.533919 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:43.534361 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 145698us
I20260709 12:26:43.464592  5664 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:43.466315  5826 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:43.462139  5721 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.519392  5831 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:43.558552 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.512341  5775 raft_consensus.cc:2804] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:43.535401 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:43.508018  5636 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.536458  5657 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:43.543926  5707 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:43.525609  5665 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:43.569339 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 112017us
I20260709 12:26:43.571027 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:43.583192  5561 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:43.550176  5734 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:43.590775 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 182414us
W20260709 12:26:43.660245 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 89319.5us
I20260709 12:26:43.665290 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:43.681790 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:43.579962  5636 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:43.589247  5834 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 12:26:43.564263  5721 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.669595  5549 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:43.682881 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:43.697162 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:43.698453  5855 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:26:43.664072  5657 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:43.580951  5835 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.296s	user 0.000s	sys 0.002s
W20260709 12:26:43.662392  5836 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 12:26:43.578640  5496 raft_consensus.cc:2804] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:43.578706  5852 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:43.702960  5833 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 12:26:43.578894  5765 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:43.707480  5842 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:26:43.700448  5805 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:43.581377  5775 raft_consensus.cc:697] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:43.669251  5734 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.729657  5575 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:43.727232  5665 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:43.752090 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.725354  5854 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:43.749688 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:43.756574 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 87443.4us
W20260709 12:26:43.757392 31063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.000s	sys 0.010s
W20260709 12:26:43.758625 31063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.010s
W20260709 12:26:43.749130  5851 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:26:43.739586  5863 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:43.742504  5765 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.749632  5805 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:43.768026  5863 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:43.795349  5863 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:43.796561  5843 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
W20260709 12:26:43.798873  5846 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.236s	user 0.000s	sys 0.000s
W20260709 12:26:43.800978  5853 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:26:43.741057  5647 consensus_queue.cc:1048] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:26:43.803856  5775 consensus_queue.cc:237] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:43.806725  5858 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:26:43.807354  5845 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
W20260709 12:26:43.808440  5857 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:43.754937  5575 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.809073  5575 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:26:43.770133  5844 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 12:26:43.766171  5378 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:43.811741  5378 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:43.812223  5378 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.763578  5665 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:43.814079  5829 log.cc:927] Time spent T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.936s	user 0.000s	sys 0.011s
I20260709 12:26:43.770823  5765 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:43.836979 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 109228us
I20260709 12:26:43.837975 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:43.844273 31690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1492) took 1326 ms. Trace:
I20260709 12:26:43.817512  5805 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:43.742475  5852 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.765874  5854 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:43.865262  5878 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:43.851413  5733 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:43.908064 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 200005us
I20260709 12:26:43.966404 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:43.969197 30958 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:43.911646 30992 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 3146) took 1337 ms. Trace:
I20260709 12:26:43.970358 30992 rpcz_store.cc:276] 0709 12:26:42.574287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.574432 (+   145us) service_pool.cc:224] Handling call
0709 12:26:42.574720 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.608375 (+ 33655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.609649 (+  1274us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.630789 (+ 21140us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.328383 (+697594us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:43.328439 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.328544 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.911059 (+582515us) raft_consensus.cc:1685] finished
0709 12:26:43.911103 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.911438 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58370,"raft.queue_time_us":26424,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":21819,"thread_start_us":664096,"threads_started":1,"wal-append.queue_time_us":777727}
W20260709 12:26:43.922673 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.923514 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1846) took 1071 ms. Trace:
W20260709 12:26:43.925599 31033 leader_election.cc:336] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Timed out: RequestConsensusVote RPC to 127.29.37.65:34803 timed out after 1.564s (SENT)
I20260709 12:26:43.975345 31067 rpcz_store.cc:276] 0709 12:26:42.851488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.851597 (+   109us) service_pool.cc:224] Handling call
0709 12:26:42.851873 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.852219 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.852243 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.858642 (+  6399us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.276164 (+417522us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:43.276227 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.276326 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.922805 (+646479us) raft_consensus.cc:1685] finished
0709 12:26:43.922869 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.923202 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":377459,"threads_started":1,"wal-append.queue_time_us":538133}
W20260709 12:26:43.976573 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51287.9us
W20260709 12:26:43.926662 31106 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.020s	sys 0.033s
W20260709 12:26:43.979343 31106 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.020s	sys 0.033s
W20260709 12:26:43.979795 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 584982us
I20260709 12:26:43.928053 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2965) took 1303 ms. Trace:
W20260709 12:26:43.929273 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 179488us
I20260709 12:26:43.982378 31135 rpcz_store.cc:276] 0709 12:26:42.624224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.634285 (+ 10061us) service_pool.cc:224] Handling call
0709 12:26:42.634558 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.651015 (+ 16457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.651046 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.931641 (+280595us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b2c000e06f8. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:26:42.932152 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.287031 (+354879us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:43.294312 (+  7281us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.294732 (+   420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.294833 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.817364 (+522531us) raft_consensus.cc:1685] finished
0709 12:26:43.927293 (+109929us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.927744 (+   451us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26612,"prepare.queue_time_us":7090,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":21150,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":46744,"spinlock_wait_cycles":9811840,"thread_start_us":354447,"threads_started":1,"wal-append.queue_time_us":521451}
I20260709 12:26:43.985587 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:43.937525 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:43.986868 31064 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:43.988040  5867 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:43.844728 31690 rpcz_store.cc:276] 0709 12:26:42.518423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.518599 (+   176us) service_pool.cc:224] Handling call
0709 12:26:42.518960 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.542681 (+ 23721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.542717 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.559037 (+ 16320us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.695214 (+136177us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:42.695252 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.695315 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.813529 (+1118214us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:43.817038 (+  3509us) raft_consensus.cc:1685] finished
0709 12:26:43.817060 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.844211 (+ 27151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":190411,"raft.queue_time_us":33672,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":26395,"spinlock_wait_cycles":205655296,"thread_start_us":133164,"threads_started":1,"wal-append.queue_time_us":210166}
W20260709 12:26:43.943811 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 105783us
I20260709 12:26:43.944059  5378 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:43.945792  5389 consensus_meta.cc:287] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.004s
I20260709 12:26:43.990726  5389 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:43.992381 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:43.993494  5867 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:43.946496  5816 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:43.947220  5033 consensus_meta.cc:287] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.717s	user 0.015s	sys 0.000s
I20260709 12:26:43.996153  5033 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:43.875988  5832 log.cc:927] Time spent T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.991s	user 0.006s	sys 0.007s
I20260709 12:26:43.998699  5733 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:43.877854  5852 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:43.878175  5854 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.001034  5665 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:43.959180  5496 raft_consensus.cc:697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:43.937448 31065 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:44.002902 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.003199  5496 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.004725 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.004949 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:43.987035  5879 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:26:44.005894 31063 raft_consensus.cc:3060] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.007715 31067 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:44.008759 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.010296 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.000353  5733 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:43.862716  5875 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:26:44.005858 31064 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 11.
I20260709 12:26:44.015009 31106 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:44.015324  5482 consensus_meta.cc:287] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.818s	user 0.006s	sys 0.010s
I20260709 12:26:44.015372 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.016247  5707 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:44.017042  5864 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:44.017369 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.018556  5561 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.020131  5877 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:44.036769  5877 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.038271 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:44.038556 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.039479 30985 raft_consensus.cc:2393] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:44.039878  5877 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.020947 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:44.040913 30983 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.040973 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:44.043630  5033 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.029188  5879 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:43.998402  5389 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.035148  5863 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.049105 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.052469 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.058261  5879 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:44.058980  5879 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.061621  5879 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.047605  5864 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:44.077991  5864 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:44.081094 31108 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.046459  5482 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.100586 31060 raft_consensus.cc:3060] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.106061 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:44.119725 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:44.121151 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:26:44.122371 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 1.10513e+06us
I20260709 12:26:44.126420 30955 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:44.128408  5879 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:44.129403  5877 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:44.130002 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.130434  5877 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:44.135809 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 48903.5us
I20260709 12:26:44.139753 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:44.133962 31068 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 1845) took 1282 ms (client timeout 1628 ms). Trace:
I20260709 12:26:44.104763  5888 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:44.174784 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.176041  5636 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.178699  5863 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:44.183176  5885 log.cc:927] Time spent T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:26:44.191339 31063 raft_consensus.cc:2468] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:44.192487 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:44.164515 31068 rpcz_store.cc:269] 0709 12:26:42.851239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.852248 (+  1009us) service_pool.cc:224] Handling call
0709 12:26:44.133534 (+1281286us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b54002cddd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:44.133749 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126707072}
I20260709 12:26:44.211882 30989 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2843) took 1531 ms. Trace:
I20260709 12:26:44.212483 31106 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:44.213788  5877 raft_consensus.cc:2804] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:44.214025  5895 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:44.214447  5877 raft_consensus.cc:697] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:44.214720  5895 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.215664  5877 consensus_queue.cc:237] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.217180  5895 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.193028  5775 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.193146  5860 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:44.227363 31058 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.236335  5636 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.212286 30989 rpcz_store.cc:276] 0709 12:26:42.642660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.642832 (+   172us) service_pool.cc:224] Handling call
0709 12:26:42.643340 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.711510 (+ 68170us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:42.711540 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.711976 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.107524 (+395548us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:43.107562 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.096878 (+989316us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b54002c1354. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:44.098935 (+  2057us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.101277 (+  2342us) raft_consensus.cc:1685] finished
0709 12:26:44.101295 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.174276 (+ 72981us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3405,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":26626,"spinlock_wait_cycles":155934080,"thread_start_us":395201,"threads_started":1,"wal-append.queue_time_us":570111}
I20260709 12:26:44.237731  5482 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.247792  5827 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:44.248276  5776 consensus_meta.cc:287] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 1.242s	user 0.018s	sys 0.016s
W20260709 12:26:44.249651  5721 consensus_meta.cc:287] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.008s
I20260709 12:26:44.264657 31058 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:44.269423 31106 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:44.225673  5888 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:44.267297 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 74609.4us
I20260709 12:26:44.274562  5893 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:44.278267  5884 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:26:44.278483  5893 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.281126  5893 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.272815  5721 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.275527  5665 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:44.299647 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.289846 30983 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:44.292343  5815 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.285063  5721 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:44.306375 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 180259us
I20260709 12:26:44.289551  5888 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.272436  5733 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.324704 31137 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:44.323972 31109 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:44.290699  5665 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:44.325839  5875 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.294303  5827 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.302030  5775 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:44.282172  5700 consensus_meta.cc:287] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.197s	user 0.012s	sys 0.006s
I20260709 12:26:44.288662  5152 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:44.291154  5886 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:44.335278 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:44.438956 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 137800us
W20260709 12:26:44.327973  5830 log.cc:927] Time spent T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.449s	user 0.000s	sys 0.012s
W20260709 12:26:44.446951  5765 consensus_meta.cc:287] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.519s	user 0.016s	sys 0.002s
W20260709 12:26:44.447965  5831 consensus_meta.cc:287] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.868s	user 0.008s	sys 0.009s
I20260709 12:26:44.330792  5776 raft_consensus.cc:2804] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:44.448498  5831 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:44.335757  5575 consensus_meta.cc:287] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.526s	user 0.005s	sys 0.012s
I20260709 12:26:44.450488  5575 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.452801  5575 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:44.339264  5695 consensus_meta.cc:287] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 1.074s	user 0.010s	sys 0.007s
I20260709 12:26:44.333636  5775 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.344134  5152 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:44.458168 31142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2967) took 1833 ms. Trace:
I20260709 12:26:44.458415 31058 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:44.343106  5700 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:44.458509 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.463404  5733 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:44.463929  5865 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.464107  5733 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.464609  5865 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.458590  5695 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.468210  5695 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.458705 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.459213 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:44.447554  5765 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.482705 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.485069 31060 raft_consensus.cc:2468] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:44.486917 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.487654 30983 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 18: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:44.331121  5827 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.459877 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:44.334112  5839 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:26:44.493456 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 53736.9us
I20260709 12:26:44.460484 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:44.501194  5865 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.501842 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:44.502393  5854 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:44.503100  5854 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:44.502586  5897 log.cc:927] Time spent T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:26:44.510108 31106 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:44.511619 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.327836  5665 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.512858 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.514441 30989 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:44.512871 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.515677 31690 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:44.491272  5776 raft_consensus.cc:697] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:44.516836  5776 consensus_queue.cc:237] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.517338 30991 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:44.518379  5854 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.520283  5815 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.521062  5815 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:44.522433 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 132238us
I20260709 12:26:44.523811  5677 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.523973 31661 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:44.524444  5888 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:44.524925  5888 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.525075  5827 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:44.525362 31661 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:44.525472  5888 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:44.525615  5827 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:44.524786  5482 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:44.526247  5827 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:44.527248 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:44.528512  5684 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.528592  5775 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:44.529145  5775 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.529626  5775 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:44.529201  5684 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.524798  5906 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.006s
I20260709 12:26:44.532049  5684 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.535665  5700 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:44.536536  5700 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:44.536863  5700 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:44.537796  5700 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.538774 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:44.538295 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:44.539247  5700 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.539426 31697 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:44.539669 31689 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:44.542496 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.545804 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:44.518324 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.547040 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.548050 31689 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 26.
I20260709 12:26:44.548197 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.548849 31143 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:44.551668 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.552302 30986 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
W20260709 12:26:44.564289 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 122889us
I20260709 12:26:44.461371 31142 rpcz_store.cc:276] 0709 12:26:42.624835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.631167 (+  6332us) service_pool.cc:224] Handling call
0709 12:26:42.631509 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.632279 (+   770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.632343 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.633155 (+   812us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.699443 (+ 66288us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:42.699481 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.699543 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.327034 (+1627491us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:44.456493 (+129459us) raft_consensus.cc:1685] finished
0709 12:26:44.457604 (+  1111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.457998 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5796,"raft.queue_time_us":139,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":13579,"spinlock_wait_cycles":232611712,"thread_start_us":65937,"threads_started":1,"wal-append.queue_time_us":226267}
I20260709 12:26:44.329130  5875 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.564823 30958 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:44.571637 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.494593  5839 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:44.573720 31109 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:44.575034 31058 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:26:44.578985 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.579702 31068 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:44.524456  5677 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.566051  5636 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:44.582345 31135 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:44.587177 30991 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:44.599452 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.600255 31063 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:26:44.605820 31697 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:44.586388  5877 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:44.609012  5877 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.609349  5877 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:44.610010  5821 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.678s	user 0.013s	sys 0.038s
W20260709 12:26:44.610430  5821 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.716s	user 0.014s	sys 0.038s
I20260709 12:26:44.582672  5033 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:44.617513  5152 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.618775 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2974) took 1666 ms. Trace:
I20260709 12:26:44.619148 31136 rpcz_store.cc:276] 0709 12:26:42.951903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.952063 (+   160us) service_pool.cc:224] Handling call
0709 12:26:42.952390 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.232079 (+279689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.618312 (+1386233us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54000947d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:44.618600 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638934272}
I20260709 12:26:44.619768  5152 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.620260 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:44.622771  5894 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.428s	user 0.007s	sys 0.000s
I20260709 12:26:44.582201  5677 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.624882 31063 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:44.625272 30987 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2834) took 2014 ms. Trace:
I20260709 12:26:44.625816 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.626418 31063 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:44.626499 31064 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:44.627626 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.627758 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.628163 31064 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 26
W20260709 12:26:44.628859 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 103602us
I20260709 12:26:44.628976 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.629645 31062 raft_consensus.cc:2368] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22.
I20260709 12:26:44.630542 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.630734  5895 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:44.631378 31067 raft_consensus.cc:2410] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:26:44.632190 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.634501  5657 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:44.635106  5657 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.635573  5875 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:44.635761 31137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42006 (request call id 1371) took 1600 ms (client timeout 1577 ms). Trace:
W20260709 12:26:44.636168 31137 rpcz_store.cc:269] 0709 12:26:43.034963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.035492 (+   529us) service_pool.cc:224] Handling call
0709 12:26:44.635541 (+1600049us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540012cfd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:44.635663 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2829938944}
I20260709 12:26:44.637207  5657 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.638708 31137 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:26:44.623966  5838 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.714s	user 0.000s	sys 0.014s
W20260709 12:26:44.639642  5813 log.cc:927] Time spent T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.724s	user 0.000s	sys 0.004s
W20260709 12:26:44.642584  5847 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.077s	user 0.002s	sys 0.000s
I20260709 12:26:44.620465  5831 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.620820 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.677294 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49530 (request call id 2842) took 2055 ms. Trace:
I20260709 12:26:44.677858 30986 raft_consensus.cc:2393] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 19: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:44.677686 30988 rpcz_store.cc:276] 0709 12:26:42.621253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.625675 (+  4422us) service_pool.cc:224] Handling call
0709 12:26:42.625993 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.685715 (+ 59722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.685794 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.962903 (+277109us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b2c0004d238. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eea18da11 00007f2eea19d372 00007f2eeb8cfa01 00007f2eeb89443a 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2
0709 12:26:42.963313 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.324711 (+361398us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:43.324768 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.324858 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.641648 (+1316790us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:44.676860 (+ 35212us) raft_consensus.cc:1685] finished
0709 12:26:44.676890 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.677076 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41515,"raft.queue_time_us":41727,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":6297,"spinlock_wait_cycles":11682816,"thread_start_us":360956,"threads_started":1,"wal-append.queue_time_us":499555}
I20260709 12:26:44.591212  5636 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.682372  5636 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:44.582531  5733 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.643533 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.656623  5839 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.657243  5811 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.658664 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:44.660401  5687 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:44.661123  5695 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:44.666642  5775 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.717058  5815 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:44.667608  5805 consensus_meta.cc:287] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Time spent flushing consensus metadata: real 0.808s	user 0.002s	sys 0.014s
I20260709 12:26:44.668109 30955 leader_election.cc:400] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22.
I20260709 12:26:44.719991 30955 leader_election.cc:403] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:26:44.670528 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:44.671906 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 177034us
I20260709 12:26:44.675040  5765 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:44.680265 31138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 2949) took 2608 ms. Trace:
I20260709 12:26:44.625625 30987 rpcz_store.cc:276] 0709 12:26:42.610302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.614311 (+  4009us) service_pool.cc:224] Handling call
0709 12:26:43.171557 (+557246us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:43.196907 (+ 25350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.624880 (+1427973us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54003453d0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8938ad 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:44.625118 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280812800}
I20260709 12:26:44.640993  5827 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.690495 31143 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:44.696892 31062 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 20: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:44.698166 31135 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:44.702991 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.730073  5665 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:44.730985  5665 consensus_queue.cc:1243] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:44.731079 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:44.734289 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.700184  5033 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:44.732867 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 64993.6us
I20260709 12:26:44.719483  5775 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.719744  5695 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:44.741112 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.741845 31067 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:44.744164  5864 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:44.744735  5864 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.746806  5864 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.750603 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.751222 31062 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.752794  5839 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:44.754253  5695 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:44.755241  5695 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.756547  5695 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:44.757455 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.761097  5890 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.558s	user 0.012s	sys 0.000s
I20260709 12:26:44.763470 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:44.764068 31693 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:44.764292 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:44.764940 31689 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 14.
I20260709 12:26:44.766098 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:44.766607 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:44.766786 31691 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.766980 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:44.767333 31697 raft_consensus.cc:2393] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:44.767490 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:44.767736 31690 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:44.769354 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:44.770615 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:44.773873  5734 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.774369 31065 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:44.775910 31691 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:44.781257 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:44.782097  5895 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:44.782495  5895 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:44.782843  5895 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:44.787177 31692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1454) took 1809 ms (client timeout 1772 ms). Trace:
W20260709 12:26:44.787611 31692 rpcz_store.cc:269] 0709 12:26:42.977429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.977574 (+   145us) service_pool.cc:224] Handling call
0709 12:26:44.786760 (+1809186us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400511b54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:44.786972 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659744128}
I20260709 12:26:44.722604  5877 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.787181  5826 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22.
I20260709 12:26:44.792706 31060 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:44.793339 31061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1852) took 1845 ms. Trace:
I20260709 12:26:44.793673 31061 rpcz_store.cc:276] 0709 12:26:42.948298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.963933 (+ 15635us) service_pool.cc:224] Handling call
0709 12:26:42.995704 (+ 31771us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:43.241337 (+245633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.792890 (+1551553us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400383e54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:44.793176 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474403584}
I20260709 12:26:44.794759 31064 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 26.
I20260709 12:26:44.800582 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.801301 30986 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:44.801702 30987 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:26:44.815168 31063 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 19.
I20260709 12:26:44.817135 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.817919 30983 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:44.818439 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:44.819852 30989 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:44.823602 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 79907.1us
I20260709 12:26:44.818668 31064 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:26:44.721747 31138 rpcz_store.cc:276] 0709 12:26:42.071304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.072976 (+  1672us) service_pool.cc:224] Handling call
0709 12:26:42.073332 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.097415 (+ 24083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.097442 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.097979 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.752403 (+654424us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b180019a920. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:26:42.818868 (+ 66465us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:42.818925 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.818990 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.623103 (+1804113us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:44.679906 (+ 56803us) raft_consensus.cc:1685] finished
0709 12:26:44.679927 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.680075 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36916,"raft.queue_time_us":23823,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5853,"spinlock_wait_cycles":161970304,"thread_start_us":66247,"threads_started":1,"wal-append.queue_time_us":811328}
I20260709 12:26:44.742933  5879 raft_consensus.cc:2764] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:44.830766 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:44.843070 30987 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:44.845945 31065 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:44.853016 31689 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:26:44.733757  5805 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.853014 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.853070 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:44.858865 30983 raft_consensus.cc:2368] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17.
W20260709 12:26:44.859387  5889 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.659s	user 0.004s	sys 0.000s
I20260709 12:26:44.860824 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.861429 31065 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:44.863073  5900 log.cc:927] Time spent T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.642s	user 0.000s	sys 0.005s
I20260709 12:26:44.864969  5827 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.866875 31061 pending_rounds.cc:85] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 15
I20260709 12:26:44.867210 31061 pending_rounds.cc:107] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:26:44.867481 31061 raft_consensus.cc:2889] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:44.871275 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3263) took 2139 ms. Trace:
I20260709 12:26:44.871701 31059 rpcz_store.cc:276] 0709 12:26:42.731481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.733518 (+  2037us) service_pool.cc:224] Handling call
0709 12:26:42.733909 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.734354 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.734394 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.734920 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.982471 (+247551us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:43.294924 (+312453us) log.cc:844] Serialized 14 byte log entry
0709 12:26:44.870463 (+1575539us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x7b180030d9c0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287
0709 12:26:44.870673 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.870772 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.870829 (+    57us) raft_consensus.cc:1685] finished
0709 12:26:44.870931 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.871068 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":462399,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":278764,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":13885,"spinlock_wait_cycles":18110336,"thread_start_us":767271,"threads_started":3,"wal-append.queue_time_us":504684}
I20260709 12:26:44.875573 31062 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:44.878942 31065 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:44.845422  5877 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.889667 31690 raft_consensus.cc:1275] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:44.890853  5033 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.892268  5888 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.893514 31696 raft_consensus.cc:1240] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:44.895442  5496 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:44.898442 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:44.899628  5496 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:44.900156  5496 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:44.907429 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.908140 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:44.908599 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.909272 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.909644 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.901176  5496 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.910506 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.910815 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.907449 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.912166 31139 raft_consensus.cc:2368] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17.
I20260709 12:26:44.913980 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:44.913637 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:44.915783 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.925911 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.929888 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:44.929967  5805 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:44.930523 30985 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:44.930670 31141 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 16.
I20260709 12:26:44.931613 30983 raft_consensus.cc:1275] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:44.936892  5888 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:44.939740 30955 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:44.924324  5528 consensus_queue.cc:1059] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:26:44.945180 31062 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:26:44.945935 31059 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:26:44.946316 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:44.949465 31138 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:44.950632 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.951213 31143 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.951242  5561 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.951805  5528 consensus_queue.cc:1243] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:26:44.952404  5528 raft_consensus.cc:3055] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:44.951946  5561 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.952768  5528 raft_consensus.cc:740] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:44.922446 31067 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:44.960104 31143 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:44.961237  5528 consensus_queue.cc:260] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:44.963291 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:44.963469 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:44.968653 30955 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:44.969313  5892 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:44.969779  5721 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:44.970515  5815 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:44.986229  5815 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.986732  5815 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.970942  5665 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:26:44.992074 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.994467 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:44.995486  5665 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:44.995981 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:44.962603 30989 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:44.982456 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:44.976763 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:44.997174 31137 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.997917  5916 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:44.998337  5916 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:44.998730  5916 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.000735  5815 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:44.972431 30987 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 16.
I20260709 12:26:44.996037  5665 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:44.984500 31062 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:45.002126  5665 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:45.002987 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:45.006677 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:45.009856 31137 raft_consensus.cc:2468] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:45.017472 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:44.965770  5895 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.021559  5721 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:45.035872 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:45.022353  5892 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
W20260709 12:26:45.061126 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 104309us
I20260709 12:26:44.961283  5877 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 12:26:45.067219 31691 raft_consensus.cc:1240] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:45.069623 30955 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:45.065197 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49081.6us
I20260709 12:26:45.022854  5805 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:45.023463  5867 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:45.075459 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.974344  5864 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:26:45.074879  5839 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.077292  5776 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:45.080449 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 63068.6us
I20260709 12:26:45.083601 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:45.080893 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45142.6us
I20260709 12:26:45.081178  5687 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:45.065783  5575 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
W20260709 12:26:45.129518 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58540.8us
I20260709 12:26:45.085611  5528 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:45.125435 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 63457.4us
I20260709 12:26:45.132059 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:45.081413  5805 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:45.121498  5561 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:45.133570 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.079375  5721 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:26:45.123844  5895 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:45.138690 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 55184.3us
I20260709 12:26:45.127360  5903 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:45.147680 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:45.131490  5687 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:45.160866  5805 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.162115  5903 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.142171  5775 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.164351  5903 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:45.155424  5892 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.165354  5687 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.166304  5630 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.153290  5496 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:45.150204  5033 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:45.173094 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:45.140527  5867 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:26:45.148036  5875 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:45.202828 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 60257.8us
I20260709 12:26:45.158421  5665 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:45.210739 30955 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.164368  5775 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.212373 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:45.172999  5636 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.201139  5837 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:45.208915  5874 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:26:45.255717 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 114466us
I20260709 12:26:45.269294 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:45.271971 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 59722.4us
W20260709 12:26:45.256826  5926 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:26:45.257237 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 84228.1us
I20260709 12:26:45.277503  5762 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:45.368709 31136 raft_consensus.cc:2393] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:26:45.353634 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 84421.7us
I20260709 12:26:45.350308  5879 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:45.252272  5815 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:45.383360 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 98651.7us
I20260709 12:26:45.252871  5837 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:45.352324  5928 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:26:45.380800 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:45.411767 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 139131us
I20260709 12:26:45.419171 30958 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:45.428387  5839 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.429221  5572 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.266753  5875 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.403155  5837 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:45.414489  5684 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:45.457623 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 76943.1us
I20260709 12:26:45.460202 31664 leader_election.cc:400] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17.
I20260709 12:26:45.460502 31664 leader_election.cc:403] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:45.417287  5879 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:45.424767  5815 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:45.464186 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:45.465317 31664 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:45.466849 31068 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:45.467669 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:45.472342 30984 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:45.480531 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 61466.4us
I20260709 12:26:45.490345 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:45.493800  5927 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:26:45.426782  5762 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.468809  5892 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.484807  5839 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:45.477901  5572 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565) with higher term. Message: Invalid argument: T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17.
I20260709 12:26:45.484474  5816 raft_consensus.cc:2804] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:45.512722 31689 raft_consensus.cc:1275] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:45.513145 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:45.480293  5684 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.528781 30958 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:45.531173  5932 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:26:45.531948 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:45.532325  5684 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:45.533011  5932 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:45.533229 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:45.533377  5684 consensus_queue.cc:1048] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:45.535791 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:45.539834 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:45.540946 31698 raft_consensus.cc:2393] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:45.533560  5932 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:45.511359  5839 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:45.551614  5839 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:45.555843  5903 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.556329  5903 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.547891  5775 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:45.564893 31058 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:26:45.521140  5816 raft_consensus.cc:697] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:45.569455  5816 consensus_queue.cc:237] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.491456  5815 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.574515  5815 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:45.564028 31664 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.577695 31664 leader_election.cc:400] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for earlier term 16. Current term is 17.
W20260709 12:26:45.566697  5916 consensus_meta.cc:287] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.568s	user 0.006s	sys 0.005s
I20260709 12:26:45.566757  5932 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.583599 31664 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:45.539628  5776 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.565635  5630 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:45.587097 30984 raft_consensus.cc:2468] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:45.593226 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:45.595242 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:45.596958 31109 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:45.602011  5630 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.585562  5916 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.604388  5916 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:45.579129  5879 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.616518 30983 raft_consensus.cc:1275] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:45.630365  5849 consensus_queue.cc:1048] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:45.632661  5601 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.632930  5839 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:45.631857 30876 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2439e7934b98446ca4e7d82381e8a499 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e9cccf7eb2a545ef8c2169b67ff261bf is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:45.620241  5877 raft_consensus.cc:2804] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:26:45.638441  5930 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.374s	user 0.001s	sys 0.001s
W20260709 12:26:45.643777 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48620.1us
I20260709 12:26:45.689818 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:45.693812 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:45.644374  5601 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:45.686875  5839 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:45.648550  5877 raft_consensus.cc:697] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:45.730068 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:45.687845  5875 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:45.745468 31137 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 19.
I20260709 12:26:45.641830  5895 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.706467  5572 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:45.707914  5849 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:45.706663  5916 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.697435  5528 consensus_queue.cc:1048] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:26:45.745121  5389 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:45.759251  5669 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:45.754194  5875 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:45.788636  5684 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:45.757256  5700 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.786746  5916 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.811890  5877 consensus_queue.cc:237] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:45.757643 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 227887us
I20260709 12:26:45.816115  5684 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.819228  5684 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:45.819761  5916 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:45.819698  5700 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.820871 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 3161) took 1689 ms. Trace:
I20260709 12:26:45.821251 30985 rpcz_store.cc:276] 0709 12:26:44.131457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.133029 (+  1572us) service_pool.cc:224] Handling call
0709 12:26:44.133472 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.941925 (+808453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.820632 (+878707us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54000221d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:45.820793 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1832600832}
I20260709 12:26:45.822096  5700 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:45.823210  5925 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.607s	user 0.000s	sys 0.000s
I20260709 12:26:45.823964 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:45.824717  5700 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:45.825438  5920 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.002s	user 0.002s	sys 0.004s
I20260709 12:26:45.826238  5700 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:45.827006 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2869) took 1286 ms. Trace:
I20260709 12:26:45.827437 31135 rpcz_store.cc:276] 0709 12:26:44.540988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.542155 (+  1167us) service_pool.cc:224] Handling call
0709 12:26:45.826826 (+1284671us) spinlock_profiling.cc:243] Waited 39.1 ms on lock 0x7b540001b154. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:45.826941 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89884928}
I20260709 12:26:45.837720 30990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1358) took 2602 ms. Trace:
W20260709 12:26:45.820904  5942 log.cc:927] Time spent T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:45.841877 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:45.844507 31690 raft_consensus.cc:2393] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 12: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:45.766597  5805 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:45.844203 30958 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:45.838124 30990 rpcz_store.cc:276] 0709 12:26:43.235046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.237914 (+  2868us) service_pool.cc:224] Handling call
0709 12:26:43.238287 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.238513 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:43.241865 (+  3352us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.426116 (+2184251us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b1800288e20. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a 00007f2eeb89ea2a
0709 12:26:45.754117 (+328001us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:45.760774 (+  6657us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:45.813035 (+ 52261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.816642 (+  3607us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:45.837540 (+ 20898us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2327,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":70441,"spinlock_wait_cycles":24440064,"thread_start_us":291819,"threads_started":1,"wal-append.queue_time_us":2549179}
I20260709 12:26:45.837965  5700 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:26:45.858832 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:45.860258 31135 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:45.862484 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:45.766983  5389 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.864264 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:45.869114 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:45.869271  5553 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.869884  5553 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.870394 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:45.758632  5849 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.871241 31689 raft_consensus.cc:1240] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:45.871523 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:45.871601 31138 raft_consensus.cc:2468] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 25.
I20260709 12:26:45.872115 31697 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:45.872536 31689 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:26:45.873175 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:45.874534 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:45.874728 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:45.870508  5389 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:45.860352  5630 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:45.879599 31693 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:45.875415 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:45.872558  5553 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:45.884392 31137 raft_consensus.cc:1240] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:45.887499 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:45.887837  5687 raft_consensus.cc:493] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:45.888360  5687 raft_consensus.cc:515] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.889060 31143 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:45.890300 31141 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:45.853852  5875 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:45.853096  5805 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:45.874176  5849 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:45.874352  5951 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.964942  5590 consensus_queue.cc:1059] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.091s
I20260709 12:26:45.968997  5687 leader_election.cc:290] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:45.992961 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 297813us
I20260709 12:26:45.996009 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:46.007133  5951 raft_consensus.cc:3055] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:26:46.008963 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 111166us
I20260709 12:26:46.013448 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:46.018337 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:46.037510 31141 raft_consensus.cc:1240] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:26:46.038640 31062 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:46.010465  5951 raft_consensus.cc:740] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:46.038968 30986 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41814 (request call id 1375) took 1137 ms. Trace:
I20260709 12:26:46.042811 30986 rpcz_store.cc:276] 0709 12:26:44.901057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.909415 (+  8358us) service_pool.cc:224] Handling call
0709 12:26:44.909696 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.909978 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:45.436452 (+526474us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.038605 (+602153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:46.038657 (+    52us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:46.038707 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.038781 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.038890 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":672262,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":42962,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":31802,"spinlock_wait_cycles":56426368,"thread_start_us":1055135,"threads_started":3,"wal-append.queue_time_us":356311}
I20260709 12:26:46.067987 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.027375  5816 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:46.012848  5630 consensus_queue.cc:1048] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:26:46.080875 31064 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:26:46.054456 31033 connection.cc:256] RPC call timeout handler was delayed by -1.8822020789998533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
W20260709 12:26:46.078442  5837 consensus_meta.cc:287] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.000s
I20260709 12:26:46.078872  5852 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:26:46.093807 31033 connection.cc:256] RPC call timeout handler was delayed by -1.8354020789997776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 12:26:46.094460 31033 connection.cc:256] RPC call timeout handler was delayed by -1.7760020789996815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 12:26:46.094888 31033 connection.cc:256] RPC call timeout handler was delayed by -1.7697020789996714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
I20260709 12:26:46.074244  5816 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:46.098955 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:46.079530  5946 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:46.048025  5879 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:26:46.133808 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.108072 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.113816  5951 consensus_queue.cc:260] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.136534 31143 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:46.097435  5816 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.116659 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.079209  5575 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.022s
I20260709 12:26:46.096563  5837 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.122524 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:46.138909 31135 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 17: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:46.132155 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:46.131969  5946 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.132728  5852 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:46.126096 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.153887  5946 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:46.160471 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.161666 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:46.162415 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56877.5us
W20260709 12:26:46.163551 31033 connection.cc:256] RPC call timeout handler was delayed by -1.28203825699984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
W20260709 12:26:46.163859 31033 connection.cc:256] RPC call timeout handler was delayed by -1.2694382569998197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
W20260709 12:26:46.164248 31033 connection.cc:256] RPC call timeout handler was delayed by -1.2311889600000541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
I20260709 12:26:46.164979 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1507) took 1626 ms. Trace:
I20260709 12:26:46.165261 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.166106  5561 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:46.166380  5553 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:46.167245 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.168473  5636 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:46.178452  5951 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:46.178289 31696 rpcz_store.cc:276] 0709 12:26:44.538407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.538826 (+   419us) service_pool.cc:224] Handling call
0709 12:26:44.539220 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.893860 (+354640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.164740 (+1270880us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54004d4ed4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:46.164908 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814123776}
W20260709 12:26:46.171543  5947 log.cc:927] Time spent T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.353s	user 0.004s	sys 0.008s
I20260709 12:26:46.175125 31692 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:46.163745 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:46.181289  5762 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:46.182816  5805 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:46.180414 31062 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:46.176291  5837 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:46.186306 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 55767.6us
I20260709 12:26:46.191979 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.196259 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.197011 31063 raft_consensus.cc:2393] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:46.149272  5389 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:46.198500 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.218358 31135 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:46.219099 31060 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:46.200383  5389 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:46.215838  5831 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.216039  5684 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:46.202069  5152 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:26:46.235152 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:46.217134  5575 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:26:46.217284  5877 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
W20260709 12:26:46.271404 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.241583  5553 raft_consensus.cc:2764] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:26:46.247112  5816 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.233760  5831 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.239005  5852 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:46.252302 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.228060  5875 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:46.262126  5932 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:46.265252  5389 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.266343  5839 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.217075  5879 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:26:46.331851 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.332937 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:46.340905 31109 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.29.37.68:35417 timed out after 1.713s (SENT) [suppressed 12 similar messages]
I20260709 12:26:46.353319  5389 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:46.346596  5152 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:46.359144  5152 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:46.362174 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 133696us
I20260709 12:26:46.365933 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.368860  5152 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:46.371627 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.367255 30988 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:46.370079 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:46.385219 31136 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45364 (request call id 2874) took 1797 ms (client timeout 2066 ms). Trace:
W20260709 12:26:46.385663 31136 rpcz_store.cc:269] 0709 12:26:44.592206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.619993 (+ 27787us) service_pool.cc:224] Handling call
0709 12:26:46.385022 (+1765029us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54003f48d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:46.385159 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508746112}
I20260709 12:26:46.389951 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:46.393146 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 116861us
I20260709 12:26:46.393304 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.397959 31059 raft_consensus.cc:2393] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 17: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:46.398696 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.402021 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.407426 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:46.409456 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.370944  5839 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.414192 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.405249  5816 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:46.417887  5152 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:46.418532  5152 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.421398 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.421615  5967 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:46.422190  5967 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.428401  5839 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:46.430606 30988 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 17.
I20260709 12:26:46.434731 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:46.436517 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 95661.3us
W20260709 12:26:46.439613 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 44896.3us
I20260709 12:26:46.430294  5636 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:46.440847  5553 raft_consensus.cc:2764] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:46.456740  5636 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.458884  5636 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:46.462594 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 72345.5us
I20260709 12:26:46.462719  5852 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:46.463310  5852 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:46.463805 31033 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.462548 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.467370 30990 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 25.
I20260709 12:26:46.471067 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13959daae3c149d6ae8248d1f3ca6e59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.469252  5553 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:46.474289 31694 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 1466) took 1710 ms. Trace:
I20260709 12:26:46.474632  5977 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:46.475188  5665 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:46.475808  5665 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:46.476286  5665 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:46.477018  5831 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:46.480486  5903 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:46.482549 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.482759  5553 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:46.483027 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.484231  5553 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:46.484290 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.485081  5553 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:46.437536  5826 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:26:46.437314  5951 consensus_queue.cc:1059] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:26:46.487346  5951 raft_consensus.cc:3055] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:46.487715  5951 raft_consensus.cc:740] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:46.488634  5951 consensus_queue.cc:260] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.489820  5951 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.466384  5700 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:26:46.492256  5700 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:46.492645  5700 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:46.474658 31694 rpcz_store.cc:276] 0709 12:26:44.764231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.764922 (+   691us) service_pool.cc:224] Handling call
0709 12:26:44.765338 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:44.765883 (+   545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:44.765912 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:44.766552 (+   640us) log.cc:844] Serialized 52 byte log entry
0709 12:26:44.985487 (+218935us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:44.985805 (+   318us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.083449 (+ 97644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.083543 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.417040 (+1333497us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b1800017ce0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:46.473878 (+ 56838us) raft_consensus.cc:1685] finished
0709 12:26:46.473971 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.474113 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48714,"prepare.queue_time_us":168,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":49330,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":5001,"spinlock_wait_cycles":72708352,"thread_start_us":210683,"threads_started":1,"wal-append.queue_time_us":323922}
I20260709 12:26:46.497371 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.493534  5700 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.508191  5960 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:46.508496  5967 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:46.516650 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:46.519876 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.512669  5700 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:46.521615 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.510658  5665 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:46.523408 30986 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 24.
I20260709 12:26:46.524873 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:46.527513 30987 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:46.511615  5960 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:46.530381  5960 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:46.530530 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.531687 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:46.539381  5561 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:46.539382  5852 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:46.548769 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.550381 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:46.551096  5553 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:46.551702 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.556365 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.556988 31692 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:46.558398  5826 consensus_queue.cc:1059] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:46.559514  5826 raft_consensus.cc:3055] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:46.559904  5826 raft_consensus.cc:740] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:46.560261 30958 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:46.560760  5826 consensus_queue.cc:260] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.561672 30958 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:46.562610  5949 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:46.562868  5932 consensus_queue.cc:1059] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:46.563700  5932 raft_consensus.cc:3055] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:46.564018  5932 raft_consensus.cc:740] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:46.564814  5932 consensus_queue.cc:260] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.565894  5932 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:46.541086  5960 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:46.567970 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:46.569226 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.569698  5960 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:46.570050 31062 raft_consensus.cc:2410] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:26:46.568540  5967 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:46.570849 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.571964 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:46.573313 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.574973 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.570866 30958 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:46.583935 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.592463 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.592682 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.577836  5826 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:46.593654 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.594426  5960 raft_consensus.cc:2749] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.594841 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.595966 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.602793 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.604370 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.607024 31136 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:26:46.610353 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.613070 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.613648 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.615324 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:46.618584 31142 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:26:46.622155 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.627178 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.628093 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.628254 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:46.602522  5553 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:46.633970 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:46.639611 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:46.640944 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:46.641528 31143 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:46.639264 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.643010 31690 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 25.
I20260709 12:26:46.643760 31062 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 25: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:46.646212 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.647751 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.649045 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:46.649833 30987 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:46.650231 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:46.652992 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.654029 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.654981 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.655874 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.656780 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:46.658088 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.658931 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.661569 30984 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 20: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:46.662572 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:46.664312 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:46.665709 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:46.669864 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:46.670451 31135 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.674661 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:46.676425 31690 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:46.657823  5978 raft_consensus.cc:2749] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:46.680156 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 128514us
I20260709 12:26:46.674093  5990 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.699123 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:46.700706 31693 raft_consensus.cc:2393] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 17: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:46.663437  5903 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:26:46.739501 31143 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:46.741124 31061 raft_consensus.cc:3055] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:46.741611 31061 raft_consensus.cc:740] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:46.754542 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51360 (request call id 1473) took 1256 ms. Trace:
I20260709 12:26:46.754949 31698 rpcz_store.cc:276] 0709 12:26:45.497772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.497907 (+   135us) service_pool.cc:224] Handling call
0709 12:26:46.754350 (+1256443us) spinlock_profiling.cc:243] Waited 9.55 ms on lock 0x7b5400514d54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:46.754488 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21966720}
I20260709 12:26:46.755707  5988 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:46.755964  5985 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:26:46.756286  5988 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:46.756765  5943 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.939s	user 0.006s	sys 0.006s
I20260709 12:26:46.771739 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:46.771857 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:46.772396 31698 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:46.772509 31137 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
W20260709 12:26:46.773507  5923 log.cc:927] Time spent T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 1.507s	user 0.016s	sys 0.000s
I20260709 12:26:46.774863 31144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1365) took 3756 ms. Trace:
I20260709 12:26:46.775655 31692 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:46.777122 31691 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:26:46.790129 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2861) took 2296 ms. Trace:
I20260709 12:26:46.789354 31135 raft_consensus.cc:2468] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:46.790530 31140 rpcz_store.cc:276] 0709 12:26:44.493753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.493907 (+   154us) service_pool.cc:224] Handling call
0709 12:26:44.555045 (+ 61138us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:44.820780 (+265735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:44.820818 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:44.840532 (+ 19714us) log.cc:844] Serialized 52 byte log entry
0709 12:26:45.183259 (+342727us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:45.208716 (+ 25457us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.253749 (+ 45033us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.278731 (+ 24982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.278809 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.782694 (+1503885us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b180049e040. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:46.789920 (+  7226us) raft_consensus.cc:1685] finished
0709 12:26:46.789945 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.790074 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":217864,"prepare.queue_time_us":29852,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":189474,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":13226,"spinlock_wait_cycles":27758848,"thread_start_us":510725,"threads_started":2,"wal-append.queue_time_us":403345}
I20260709 12:26:46.754071 31061 consensus_queue.cc:260] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:46.775209 31144 rpcz_store.cc:276] 0709 12:26:43.018467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.018633 (+   166us) service_pool.cc:224] Handling call
0709 12:26:43.019005 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:43.019390 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.019429 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:43.019964 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:26:44.862893 (+1842929us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b1800363400. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8789e07 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb4340d1 00007f2eeb84fa0f
0709 12:26:45.089805 (+226912us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:45.089842 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.090009 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.772852 (+1682843us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:46.774552 (+  1700us) raft_consensus.cc:1685] finished
0709 12:26:46.774572 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.774710 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21554,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"spinlock_wait_cycles":86172160,"thread_start_us":226606,"threads_started":1,"wal-append.queue_time_us":2121058}
I20260709 12:26:46.806931 31068 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:26:46.806859 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 120490us
I20260709 12:26:46.708228  5990 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:46.794107  5991 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:26:46.811021 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 129058us
I20260709 12:26:46.813464 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.815052 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:46.816157  5839 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:46.816570  5839 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.817003  5839 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:46.824887  5987 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:26:46.797206  5775 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.854031  5990 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:46.828632  5991 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:46.855808  5991 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:46.861763 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 199252us
I20260709 12:26:46.862573 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:46.863420 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:46.864924  5950 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:46.865665  5950 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:46.861824  5852 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:46.869266  5852 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:46.869657  5852 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:46.870410 31061 raft_consensus.cc:3060] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:46.874290  5963 log.cc:927] Time spent T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.731s	user 0.002s	sys 0.002s
I20260709 12:26:46.875353 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.876847  5496 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:46.877020 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 63662.7us
I20260709 12:26:46.878108 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:46.878449 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.879683 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:46.884308  5496 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:46.884689  5988 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:46.884737  5496 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:46.885471  5496 raft_consensus.cc:3060] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:46.885298  5950 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:46.888262 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:46.889011  5950 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:46.889371  5950 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:46.891464 30986 raft_consensus.cc:2468] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 15.
I20260709 12:26:46.889725  5950 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:46.898823 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.900556 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.906879 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.908277 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.910569 31109 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.882550  5852 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:46.912024 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 82322us
I20260709 12:26:46.912261 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.869794  5152 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:46.908185  6001 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:26:46.915295 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:46.944450 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:46.905296  5988 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:46.907425  5879 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:46.962774 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 47550.7us
I20260709 12:26:46.925671 31664 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:46.898547  5992 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:46.911114  5916 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:46.913477  5572 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:46.909310  5849 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:46.964093 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 47006.9us
I20260709 12:26:47.000324 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:46.969569 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 260910us
I20260709 12:26:46.940212  5389 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:46.909420  5636 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:26:46.917578  5665 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:47.010670 31061 raft_consensus.cc:1275] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
W20260709 12:26:46.986142  6004 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 12:26:47.018168 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 101352us
I20260709 12:26:47.018278 31061 pending_rounds.cc:85] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 15
I20260709 12:26:47.021250 31061 pending_rounds.cc:107] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:26:47.021535 31061 raft_consensus.cc:2889] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:47.025466 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 61984.9us
I20260709 12:26:47.026180 31109 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.027300 31109 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.043372 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:47.048391 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 131646us
I20260709 12:26:47.005780  5561 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:26:47.052361 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.010114  5879 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:47.003825  5988 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:47.020506  5837 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:47.017369  5959 raft_consensus.cc:2804] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:47.060649 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 57806.5us
I20260709 12:26:47.057868 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:47.066784 31141 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:47.074594  5918 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.168s
I20260709 12:26:47.042379  5665 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.077996  5665 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:47.058151  5999 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:47.080526  5665 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:47.082556  5837 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:47.083487  5837 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:47.085460  5995 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 12:26:47.091012 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.050304  5389 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:47.102373 31109 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:47.089869  5849 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:47.140153 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 77539.9us
W20260709 12:26:47.140277 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 234022us
I20260709 12:26:47.096213  5959 raft_consensus.cc:697] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:47.081998  5839 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.106619  5918 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:47.154419 30989 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:47.155910 30985 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
I20260709 12:26:47.106962  5990 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:47.138549  6010 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:47.164366  5849 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:47.165360 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:47.149309  5669 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.151316  6011 raft_consensus.cc:2804] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:47.160668  5990 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:47.166263  5895 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.185863 30992 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:26:47.195291  6009 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:47.189392  6011 raft_consensus.cc:697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:47.178539  5669 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.196728  6011 consensus_queue.cc:237] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.198396  5669 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:47.183985  5990 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:47.218263 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:47.186456  5895 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.179209  5988 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:47.217061  5553 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:26:47.229928  6016 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:26:47.247400  5553 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:47.247896  5553 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.248251  5553 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:47.245885  5990 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:47.250964  5990 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:47.253793  5895 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:47.261328  6010 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:47.247413 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.221671  5959 consensus_queue.cc:237] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:47.268695 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 78582.5us
I20260709 12:26:47.276180 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.277118  6000 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:47.277642 31033 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.281481  6000 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:47.283371  6000 consensus_queue.cc:1059] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:47.284008  6000 consensus_queue.cc:1243] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:47.284452  6000 raft_consensus.cc:3055] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:47.284746  6000 raft_consensus.cc:740] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:47.284829 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:47.285552 31139 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:47.284943  5950 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:47.285593  6000 consensus_queue.cc:260] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:47.286937  6000 raft_consensus.cc:3060] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:47.290794 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:47.291502 30986 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:47.300803  5950 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:47.303175 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:47.303994 31696 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 22: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
W20260709 12:26:47.303648 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 57435us
I20260709 12:26:47.301079 30986 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 25.
I20260709 12:26:47.309195 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:47.313920 31061 raft_consensus.cc:2410] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 23 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:26:47.310029 31065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:47.315063 31065 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:47.325640 31065 raft_consensus.cc:2410] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:26:47.336258  5553 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.312822  5669 consensus_queue.cc:1048] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:26:47.438068  5572 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:47.440136  5572 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.442282  6017 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:47.443045  6017 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.443589 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:47.444903 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:47.445719  6017 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:47.452742 31136 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:47.449793 31141 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:26:47.451274 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:47.464915 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:47.468802 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:47.454403  5572 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:47.338863  5839 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:47.454492  5669 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:47.531373 31031 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.555724 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.494592  6015 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:47.563485  6030 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:47.568524  5852 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:47.570693 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:47.571944  6030 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:47.558902  5496 consensus_meta.cc:287] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Time spent flushing consensus metadata: real 0.671s	user 0.005s	sys 0.009s
W20260709 12:26:47.585713 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 117877us
W20260709 12:26:47.594671 30958 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.597884 31033 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.602423 31033 leader_election.cc:304] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.603502 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:47.604291  6000 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:47.604940  6000 raft_consensus.cc:2749] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:47.605480  5990 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:47.605897  5990 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.608278 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 518047us
I20260709 12:26:47.595893  5496 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.610992  5496 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:47.612416  5990 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:47.616441  6014 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.553s	user 0.003s	sys 0.000s
W20260709 12:26:47.618093  6002 log.cc:927] Time spent T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.659s	user 0.003s	sys 0.006s
I20260709 12:26:47.621565 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3306) took 1026 ms. Trace:
I20260709 12:26:47.585306  6027 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.623004  6027 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.576611  5389 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:47.559787  5669 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:47.663287  6025 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:26:47.664490  5669 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:47.664718  5826 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.665201  5826 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:47.666481 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3314) took 1069 ms. Trace:
I20260709 12:26:47.666863 31059 rpcz_store.cc:276] 0709 12:26:46.597383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.908339 (+310956us) service_pool.cc:224] Handling call
0709 12:26:46.948156 (+ 39817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.964686 (+ 16530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.964716 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.017736 (+ 53020us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.225733 (+207997us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:47.225771 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.225840 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.666277 (+440437us) raft_consensus.cc:1685] finished
0709 12:26:47.666298 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.666427 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18457,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":33857,"thread_start_us":136433,"threads_started":1,"wal-append.queue_time_us":265783}
I20260709 12:26:47.668561 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:47.669044 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:47.671279  5721 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.671787  5721 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.672232  6027 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:47.676450 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:47.577207  6015 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.621932 31068 rpcz_store.cc:276] 0709 12:26:46.595303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.836839 (+241536us) service_pool.cc:224] Handling call
0709 12:26:46.910129 (+ 73290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.910583 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.910616 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.914283 (+  3667us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.059192 (+144909us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:47.059230 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.059290 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.615749 (+556459us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:47.621292 (+  5543us) raft_consensus.cc:1685] finished
0709 12:26:47.621313 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.621511 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2199,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":10033,"spinlock_wait_cycles":72140672,"thread_start_us":53319,"threads_started":1,"wal-append.queue_time_us":95565}
I20260709 12:26:47.679067  6015 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:47.680078  5991 consensus_meta.cc:287] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.824s	user 0.013s	sys 0.000s
I20260709 12:26:47.680496  5991 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:47.680795  5805 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:47.681231  5805 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.681586  5805 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:47.682660 31058 raft_consensus.cc:1240] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:47.682796 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:47.683899 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:47.684937 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:47.669286 30986 raft_consensus.cc:2393] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 15: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:47.686448  5553 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:47.673902  5721 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:47.695168 31138 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:47.690317 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:47.697041 31109 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:47.694229 31691 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:47.698436 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:47.722512 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:47.699637  5831 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:26:47.724924 30985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49522 (request call id 3182) took 1208 ms. Trace:
I20260709 12:26:47.725394 30985 rpcz_store.cc:276] 0709 12:26:46.516105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.516410 (+   305us) service_pool.cc:224] Handling call
0709 12:26:47.724757 (+1208347us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54003458d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:47.724846 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204097920}
I20260709 12:26:47.726878  5852 raft_consensus.cc:2804] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:47.727581  5852 raft_consensus.cc:697] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:47.728694  5852 consensus_queue.cc:237] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:47.743891 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 45516.9us
I20260709 12:26:47.746740 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:47.749655 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:47.751720 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:47.752276 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:47.757576 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:47.758155 31139 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:47.759994 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:47.762269 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:47.775612 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:47.782596 31067 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3311) took 1185 ms. Trace:
I20260709 12:26:47.783063 31033 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:47.718108  5776 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:47.782960 31067 rpcz_store.cc:276] 0709 12:26:46.596688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.859955 (+263267us) service_pool.cc:224] Handling call
0709 12:26:46.860341 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.860579 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:46.860942 (+   363us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.565250 (+704308us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b180030d660. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:47.720252 (+155002us) spinlock_profiling.cc:243] Waited 8.27 ms on lock 0x7b180030d660. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87752a6 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b
0709 12:26:47.722154 (+  1902us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b180030d660. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a6c02 00007f2eeb42e20b 00007f2eeb433d23 00007f2eeb43423a
0709 12:26:47.745087 (+ 22933us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.782229 (+ 37142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:47.782285 (+    56us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:47.782342 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.782421 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.782539 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43597,"prepare.queue_time_us":36377,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":267,"spinlock_wait_cycles":86233856,"thread_start_us":96136,"threads_started":1,"wal-append.queue_time_us":98848}
I20260709 12:26:47.792272 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:47.805742 31061 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 20.
I20260709 12:26:47.787653 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:47.721861  5990 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.825778 31109 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:47.741677  5874 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:47.748365  5152 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:47.839097 31062 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:47.836771 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:47.699324  5687 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 12:26:47.775257  5669 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:47.699968  5949 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 12:26:47.759116  5960 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:26:47.835546  5987 log.cc:927] Time spent T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:47.810281  5999 raft_consensus.cc:2764] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:26:47.804651  5553 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:47.833545  5988 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:47.811842  5991 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:47.838819  5895 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:47.848629  5665 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:47.748186  5892 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.164s
W20260709 12:26:47.961344 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 174813us
I20260709 12:26:47.934158  5864 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.938081  5496 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:26:47.972895 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 167901us
I20260709 12:26:47.849037  5992 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.114s
I20260709 12:26:47.945982  5553 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:47.975438 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:47.976039 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 139350us
I20260709 12:26:47.976514 31694 raft_consensus.cc:3060] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:47.968695 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 210131us
I20260709 12:26:47.912420  5561 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:26:47.912024  5776 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.936311  5669 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:47.970249  5826 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:47.977058  5665 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:47.980686  5553 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:47.982997  5496 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:26:48.013279  6007 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.448s	user 0.002s	sys 0.000s
W20260709 12:26:48.015388  6037 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:26:48.011910  6013 log.cc:927] Time spent T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.919s	user 0.000s	sys 0.015s
I20260709 12:26:48.000765  5561 consensus_queue.cc:1243] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer ce27a72886964e4287059a1d66bd5b6b log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:47.977782  5864 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.023038  5934 raft_consensus.cc:493] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.031965  5934 raft_consensus.cc:515] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:48.043236 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 354698us
W20260709 12:26:48.068940  6036 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:26:48.069131 31693 raft_consensus.cc:2468] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 25.
I20260709 12:26:48.071391  5934 leader_election.cc:290] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.074265  5496 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.078758  5990 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.085155  5805 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:48.087085  6038 log.cc:927] Time spent T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:48.088047  5553 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.090500  5665 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.094954  5864 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:48.096083  5776 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:47.487982 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:48.088429  5669 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:48.110456 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 267449us
W20260709 12:26:48.102522  6005 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.092s	user 0.004s	sys 0.000s
I20260709 12:26:48.117488 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:48.106323 31139 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
W20260709 12:26:48.108695 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 60658.2us
I20260709 12:26:48.090152  5553 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:48.119336  6041 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:26:48.127526 30955 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:48.128317 31694 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:48.132169 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:48.132874 30985 raft_consensus.cc:2368] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22.
I20260709 12:26:48.134054 30985 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:48.134944 30983 raft_consensus.cc:1240] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:26:48.136190 30992 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:26:48.140653 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.141112 30992 raft_consensus.cc:1240] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:48.119743  5776 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.142187 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.142724 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:48.130432  5990 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:48.142336  5695 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:48.128799  5949 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:26:48.190060 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:48.211380 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 69504.4us
I20260709 12:26:48.137428  5852 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.198625 31058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3299) took 1623 ms. Trace:
I20260709 12:26:48.202600 31141 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:48.214799 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:48.215360 30984 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:48.203380  6010 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.204063 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:48.144634  6044 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.205022 31033 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:48.217609 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:48.206115 30988 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49522 (request call id 3204) took 1059 ms. Trace:
I20260709 12:26:48.207912 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:48.214444 31058 rpcz_store.cc:276] 0709 12:26:46.575659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.769745 (+194086us) service_pool.cc:224] Handling call
0709 12:26:46.770039 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.683033 (+912994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.198370 (+515337us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54000c7954. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:48.198562 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098452736}
I20260709 12:26:48.151412  5776 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.218317 30988 rpcz_store.cc:276] 0709 12:26:47.146308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.154536 (+  8228us) service_pool.cc:224] Handling call
0709 12:26:47.167922 (+ 13386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.616258 (+448336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.616292 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.616777 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.842820 (+226043us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:47.842872 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.842933 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.205910 (+362977us) raft_consensus.cc:1685] finished
0709 12:26:48.205933 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.206056 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":447925,"raft.queue_time_us":449113,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"thread_start_us":225672,"threads_started":1,"wal-append.queue_time_us":348173}
I20260709 12:26:48.210466  6051 raft_consensus.cc:493] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.210040  5695 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.223424 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:48.223640 31033 leader_election.cc:400] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22.
I20260709 12:26:48.226609  5695 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:48.227854  6008 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 1.134s	user 0.002s	sys 0.000s
I20260709 12:26:48.227614  5553 consensus_queue.cc:1059] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:48.230518 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.216841  5852 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.231163 30990 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:26:48.232231  5992 consensus_queue.cc:1059] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:48.220999  5990 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:48.234967 30987 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:26:48.239028  6051 raft_consensus.cc:515] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.244038  5988 raft_consensus.cc:3055] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:48.244433  5988 raft_consensus.cc:740] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:48.245342  6051 leader_election.cc:290] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.243614  6030 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:48.244612  5852 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:48.221558  6010 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.221127  6044 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.245237  5988 consensus_queue.cc:260] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.258585 30984 raft_consensus.cc:2468] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:48.234807  6056 consensus_queue.cc:1059] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:26:48.239327  5805 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.248602  5990 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
W20260709 12:26:48.291849 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 61412.3us
I20260709 12:26:48.293097 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.294441 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:48.261139  6003 log.cc:927] Time spent T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 1.297s	user 0.002s	sys 0.004s
I20260709 12:26:48.298188  5988 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:48.301007  6029 log.cc:927] Time spent T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.987s	user 0.002s	sys 0.000s
I20260709 12:26:48.294757  6051 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.315304  6051 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.296332  5553 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:48.317829  5553 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:48.318429  6051 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.307160 31136 raft_consensus.cc:1240] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:48.319167  5776 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.319916  5990 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.332502  6010 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:48.337728 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.273265  6030 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.339759 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.339884  6030 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.340817 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.342193  6030 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.343660  6044 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.347872  5852 consensus_queue.cc:1059] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:48.348930  5852 raft_consensus.cc:3055] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:48.349273  5852 raft_consensus.cc:740] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:48.350147  5852 consensus_queue.cc:260] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.351644  5852 raft_consensus.cc:3060] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:48.372025 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.378561 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:48.383607  6060 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:26:48.305229  5695 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.390412  5932 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.400750 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 106369us
I20260709 12:26:48.401845 31033 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.402870  5990 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:48.404515  5990 consensus_queue.cc:1059] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:48.405789  5990 raft_consensus.cc:3055] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:48.406255  5990 raft_consensus.cc:740] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:48.407404  5990 consensus_queue.cc:260] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.408727  5990 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:48.411309  6032 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.914s	user 0.008s	sys 0.000s
I20260709 12:26:48.408309  5932 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.417438  5932 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.418789 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.419593  5561 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:26:48.419899  6000 raft_consensus.cc:3055] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:48.420295  6000 raft_consensus.cc:740] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:48.421137  5561 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:48.421168  6000 consensus_queue.cc:260] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.423382 31033 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:48.424150  6006 raft_consensus.cc:2804] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:48.424655  6006 raft_consensus.cc:697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:48.425768  5695 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.425494  6006 consensus_queue.cc:237] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:48.426608 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.430748  5990 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:48.436502 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.437335 31062 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:48.438683 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.440026 31059 raft_consensus.cc:2410] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:26:48.444622 31067 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 24. Current term is 25. Ops: []
W20260709 12:26:48.445389  6012 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 1.388s	user 0.001s	sys 0.002s
W20260709 12:26:48.447198 30881 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 202) took 37344 ms (client timeout 15000 ms). Trace:
I20260709 12:26:48.438632  6018 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.449138  6018 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.451303  5991 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:48.451973  5991 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:48.453153  5991 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.454831 31064 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3310) took 1858 ms. Trace:
I20260709 12:26:48.455166 31064 rpcz_store.cc:276] 0709 12:26:46.596442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.852827 (+256385us) service_pool.cc:224] Handling call
0709 12:26:46.853274 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.853824 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.853880 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.895492 (+ 41612us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.025536 (+130044us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:47.042089 (+ 16553us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.057818 (+ 15729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.057898 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.444738 (+1386840us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:48.454536 (+  9798us) raft_consensus.cc:1685] finished
0709 12:26:48.454638 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.454775 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20123,"prepare.queue_time_us":16551,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":20866,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":33974,"spinlock_wait_cycles":87005952,"thread_start_us":74443,"threads_started":1,"wal-append.queue_time_us":142440}
W20260709 12:26:48.456578  6007 log.cc:927] Time spent T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.415s	user 0.001s	sys 0.005s
I20260709 12:26:48.457485 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.434806 30888 catalog_manager.cc:5697] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 4 to 12, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 12 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:48.458484 30988 raft_consensus.cc:2393] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 25: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:48.460433  6039 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.495s	user 0.001s	sys 0.001s
I20260709 12:26:48.460201 30888 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 12 to 14, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 14 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:48.454658  5999 raft_consensus.cc:3055] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:48.463544  5999 raft_consensus.cc:740] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:48.464785  6000 raft_consensus.cc:3060] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:48.464414  5999 consensus_queue.cc:260] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.467563  5999 raft_consensus.cc:3060] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:48.451503  6018 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:48.470938  5553 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.471489  5553 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.479025  5879 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.479606  5879 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.480309  5934 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.480811  5934 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.481691  5879 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.478279 30888 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 17 to 19, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 19 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:48.484105  5939 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.484622  5939 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.485733 30888 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 8 to 16, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 16 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:48.486625 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 511365us
I20260709 12:26:48.499008 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.500097 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:48.500629 31138 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:48.496760 31136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.502065 31062 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:48.502588 31136 raft_consensus.cc:2410] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:26:48.493016 31109 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:48.510987 31059 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:48.526783 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.527601 31139 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:48.534332 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.535097 30988 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
W20260709 12:26:48.536429 31030 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.538003 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:48.538692 31064 raft_consensus.cc:2368] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24.
I20260709 12:26:48.501870  5879 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.539852 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:48.540380 31694 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:48.540516 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:48.509665  5839 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:48.542292 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.543457 31058 raft_consensus.cc:1275] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:26:48.543810 31058 pending_rounds.cc:85] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 12
I20260709 12:26:48.544035 31058 pending_rounds.cc:107] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:26:48.544250 31058 raft_consensus.cc:2889] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:48.544545 31058 pending_rounds.cc:107] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:26:48.544772 31058 raft_consensus.cc:2889] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:48.547145 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:48.544623 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.549679 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.551393 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.545898 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.514607  6016 raft_consensus.cc:2804] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:48.553808 31068 raft_consensus.cc:2393] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 25: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:48.573997 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:48.573580  5553 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.592681  5669 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.593497  6052 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.594039  6052 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.559783  5879 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.596336  6052 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:48.596958  6058 raft_consensus.cc:493] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.597304  5879 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.597528  6058 raft_consensus.cc:515] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.598790 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.593283  5669 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.599809  6058 leader_election.cc:290] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.600785  5669 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:48.601509  5978 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.601965  5978 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.602355  5934 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:48.602480  6020 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.560940  5496 raft_consensus.cc:493] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.603060  6020 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.603423 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.603192  5496 raft_consensus.cc:515] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.604205 31060 raft_consensus.cc:2410] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:26:48.604549 31696 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51360 (request call id 1489) took 1274 ms. Trace:
I20260709 12:26:48.604843  5528 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.605589  5528 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.605104  6020 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:48.607630  5528 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:48.594127  5998 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.515738  5665 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:26:48.612399  5998 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.614475  5998 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:48.610446 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 181353us
I20260709 12:26:48.620710 30987 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:48.623718 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.624275 30987 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:48.604900 31696 rpcz_store.cc:276] 0709 12:26:47.330363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.330495 (+   132us) service_pool.cc:224] Handling call
0709 12:26:47.330782 (+   287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:47.403039 (+ 72257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.403080 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:47.453807 (+ 50727us) log.cc:844] Serialized 52 byte log entry
0709 12:26:47.688650 (+234843us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:48.459512 (+770862us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:48.603887 (+144375us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b18002bdfe0. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2eeb43c4bc 00007f2ee878a48b 00007f2ef15e49a9 00007f2ee87a669b 00007f2ee87a493f 00007f2ee87a4455 00007f2eef28c3c7 00007f2eeb89e9a7 00007f2eeb8ae0bf
0709 12:26:48.604019 (+   132us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.604235 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.604298 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.604340 (+    42us) raft_consensus.cc:1685] finished
0709 12:26:48.604351 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.604496 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":274406,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":2730,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":168,"spinlock_wait_cycles":20594816,"thread_start_us":311947,"threads_started":2,"wal-append.queue_time_us":323951}
I20260709 12:26:48.630370 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.632473 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.633216 31061 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:48.628763  6000 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:48.642241 31061 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:48.643469 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.644074 30986 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:48.644541 31139 raft_consensus.cc:1240] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:48.605263  5496 leader_election.cc:290] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.645102 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.645476 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.650661 31696 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 24: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:48.608022  5888 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:26:48.656122 31693 raft_consensus.cc:1240] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:48.657434 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.658175 31690 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:48.658798 31063 raft_consensus.cc:2468] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 25.
I20260709 12:26:48.667037 31068 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:26:48.643085  5999 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:48.668944 31694 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:48.672144 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:48.646461  5839 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.651567  5528 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.676291 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:48.688268 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.694350 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.695171 31106 leader_election.cc:400] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24.
I20260709 12:26:48.697443 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.643049  6000 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.690622  5553 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:48.708058 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:48.708312  5553 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.708770  5553 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:48.712535 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.713681 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.553287  6046 consensus_queue.cc:1059] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.143s
I20260709 12:26:48.716802 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ff17d7bb34443adb04e437ed0079e51" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.716979 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.723294 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.711095 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.724531 31033 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:48.722470 31695 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:48.727000 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.727684 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.732177 31138 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:48.732932 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:48.727569 31695 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:48.733503 31137 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:48.735020 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 154791us
I20260709 12:26:48.738241 31690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8ba55d3e2574f7a890f7a956ba4e74f" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.739509 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.711333  5999 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:48.704159  6010 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:48.790567 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:48.791424 31689 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:48.793550 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.794207 31138 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
W20260709 12:26:48.794878 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 47889.7us
I20260709 12:26:48.733323  6027 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:48.693802  5839 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.728682  6061 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:48.734853  5852 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.795029 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.800623 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.705817  5528 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.710436  6011 consensus_queue.cc:1048] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 12:26:48.738449  5939 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.746841  5999 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:48.738245  6016 raft_consensus.cc:697] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:48.816759 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.821348 31137 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:48.812909 30958 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:48.832453 31693 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:48.807832  5888 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.813313  6027 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:48.828848  5999 raft_consensus.cc:3060] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:48.844580 31689 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:48.816573  6061 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.849941  6061 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.869640 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.850574 31693 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
W20260709 12:26:48.850730  6070 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:48.850904 30958 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:48.796484  6010 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:48.852090  6071 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:26:48.877357 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 132253us
I20260709 12:26:48.823263  5852 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.853433  5999 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.882077 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:48.859822  5839 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:48.448465 30881 rpcz_store.cc:269] 0709 12:26:11.102609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.102798 (+   189us) service_pool.cc:224] Handling call
0709 12:26:48.446753 (+37343955us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:40.524580 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:40.540228 (+ 15648us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:40.540264 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:26:40.540281 (+    17us) tablet.cc:662] Decoding operations
0709 12:26:40.541598 (+  1317us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:40.541659 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:40.541688 (+    29us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:26:40.673698 (+132010us) tablet.cc:701] Row locks acquired
0709 12:26:40.673722 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:26:40.673882 (+   160us) write_op.cc:270] Start()
0709 12:26:40.674012 (+   130us) write_op.cc:276] Timestamp: P: 1783600000673827 usec, L: 0
0709 12:26:40.674036 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:26:40.674756 (+   720us) log.cc:844] Serialized 5854 byte log entry
0709 12:26:41.010320 (+335564us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:41.695311 (+684991us) op_driver.cc:464] REPLICATION: finished
0709 12:26:42.266899 (+571588us) write_op.cc:301] APPLY: starting
0709 12:26:42.266995 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:42.576150 (+309155us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:42.576177 (+    27us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:43.754713 (+1178536us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:43.802732 (+ 48019us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 12:26:43.802782 (+    50us) write_op.cc:312] APPLY: finished
0709 12:26:43.803138 (+   356us) log.cc:844] Serialized 139 byte log entry
0709 12:26:43.947665 (+144527us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:43.970473 (+ 22808us) write_op.cc:454] Released schema lock
0709 12:26:43.970617 (+   144us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:43.971322 (+   705us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:43.971372 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":333047,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":571306,"apply.run_cpu_time_us":57656,"apply.run_wall_time_us":1704326,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66857,"num_ops":15,"prepare.queue_time_us":278111,"prepare.run_cpu_time_us":10887,"prepare.run_wall_time_us":193166,"raft.queue_time_us":679787,"raft.run_cpu_time_us":72422,"raft.run_wall_time_us":436310,"replication_time_us":1021110,"spinlock_wait_cycles":75667328,"thread_start_us":1384696,"threads_started":5,"wal-append.queue_time_us":262017,"wal-append.run_cpu_time_us":21158,"wal-append.run_wall_time_us":2723788}]]}
I20260709 12:26:48.871588 31137 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:48.873016 31141 raft_consensus.cc:1275] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:48.874993 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.884073 30958 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:48.884757 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:48.875218 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.885375 30958 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:48.885998  5892 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:48.886449  5892 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.836179  5949 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:48.888123 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.889367 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.889885 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:48.852516 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.827893  5934 consensus_queue.cc:1059] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:26:48.857129  5553 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.878324 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.886838  5932 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:48.843747  5888 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.894905 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.877086  5831 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:48.886837  5892 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:48.907774  5528 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:48.900378  6023 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:48.900936  6022 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:48.909926  6050 raft_consensus.cc:3055] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:48.910115  6022 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:48.910435  6050 raft_consensus.cc:740] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:48.910655  6022 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:48.902109  6010 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.903577  5553 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.904142  6034 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:48.905644  5849 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:48.906119  5932 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:48.906347 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.900419  5665 raft_consensus.cc:3055] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:48.914443  5665 raft_consensus.cc:740] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:48.914553  6000 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:48.906908  5831 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:48.887027  5949 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.917140 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:48.921371 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:48.913076  5932 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.924584 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:48.928470 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.928670 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.929541 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.930634 30990 raft_consensus.cc:2410] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:26:48.898552 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.936300 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:48.936957 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.940338  5496 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:48.942548  5695 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.942865 30988 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:48.943076  5695 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.945152  5695 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.946000 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.946501 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:48.946691  5978 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:48.921152  5590 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.947077 30986 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:48.947328  5590 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:48.904158  5888 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:48.949270  5991 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:48.949482  5590 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:48.950182  6072 log.cc:927] Time spent T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:48.952951 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.955646 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:48.956285 31058 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:48.957617 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "205fb3f3a3844073a49da13542da2989" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.958698 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.958738 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.961720 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:48.965642 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.959339 31067 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 12.
I20260709 12:26:48.967665 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.921319  5949 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.979389 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.977540 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.966261 30955 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:48.975551 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.965055  5528 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:48.972522  6027 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.983064 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.983695 31143 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:48.908778  6016 consensus_queue.cc:237] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.921216  5665 consensus_queue.cc:260] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.987200  5665 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:48.988211  6071 log.cc:927] Time spent T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:48.911393  6050 consensus_queue.cc:260] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.17, Last appended by leader: 17, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.950129  5590 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.973241  6000 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:26:48.990883  5999 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.991155  5590 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:48.992549 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.993193  5590 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:48.922453  6034 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:48.995648 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:48.995582 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:48.974231  6068 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 12:26:48.982936  5528 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:49.001449 30986 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:48.964505  5839 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:49.005836 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:49.005936 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:48.994753 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:48.996462 31689 raft_consensus.cc:2393] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 24: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:48.996292 30984 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
W20260709 12:26:49.005827 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.007006 31143 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 21: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:49.000953  5590 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:48.993389  6022 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.011049  6022 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.013055 30992 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:49.013625  5496 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:49.002758  5528 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.014926 31697 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:49.015390 31139 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 25.
I20260709 12:26:49.015717  5831 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.018069  6034 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:49.019083  6034 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.007360 31067 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 12.
I20260709 12:26:49.020957 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:49.025561 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.026752 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.027767 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.029721 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:49.028563 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:49.032021 30988 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
I20260709 12:26:49.029793 31058 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
W20260709 12:26:49.032261  6073 log.cc:927] Time spent T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:26:49.020193  6011 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.018151  5852 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.057256 30992 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:49.001835  5951 consensus_queue.cc:1059] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:26:49.131157 30986 raft_consensus.cc:1275] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:26:49.135005 31136 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.009s	sys 0.029s
I20260709 12:26:49.036109  5998 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:49.136672 31136 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.009s	sys 0.030s
I20260709 12:26:48.999557  5892 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.041788  5669 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:49.038348  5978 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:26:49.238517 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 101059us
I20260709 12:26:49.238771  5816 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:49.239933 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:49.240221  5816 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:49.240749  5816 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:49.250630  6023 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:49.256171 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:49.258760 31137 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.157668  5700 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:49.137115  6011 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.260039  6000 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:49.266536 31032 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.241813  5816 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.261270  6023 raft_consensus.cc:493] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.260607  6000 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.261137  5700 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:49.307190  6023 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.309757  6052 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:49.324781  6052 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:49.315527  5816 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:49.322108  6091 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:49.323475  5892 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.324296  6011 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.326006  6052 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.332791  6052 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:49.371403 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 371006us
I20260709 12:26:49.301200 31137 raft_consensus.cc:2468] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:49.374131 30955 leader_election.cc:304] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:49.377115  5999 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:49.414049 30955 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:49.357272  5949 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.435159 30986 raft_consensus.cc:1275] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:26:49.440711  5152 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:49.378041  5528 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.457829  5528 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.420986  6062 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:49.461990 31694 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:49.462092  6062 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.464278  6062 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.409062  6058 raft_consensus.cc:2749] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:49.466898 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.452178  6000 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.469458 30987 raft_consensus.cc:2393] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 13: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:49.469875 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:49.408004  6027 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.455624  5949 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.370416  5700 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.476898 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.473065  6052 consensus_queue.cc:1048] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:26:49.481489  6052 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:49.482968  5991 consensus_meta.cc:287] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.002s
I20260709 12:26:49.483343 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:49.483815 31065 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:26:49.483924 31694 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:26:49.484189 31065 pending_rounds.cc:85] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 12
I20260709 12:26:49.484432 31065 pending_rounds.cc:107] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:26:49.484650 31065 raft_consensus.cc:2889] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:49.485651  6085 log.cc:927] Time spent T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:26:49.436470  5999 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.488096 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bb92178abcc4ef385f771bd04448a5b" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:49.488999  5999 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.474424  5528 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:49.490417 31692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:49.493834 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:49.490329 31698 raft_consensus.cc:1240] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:49.497977 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.516004 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
W20260709 12:26:49.519905 30983 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.001s	sys 0.060s
I20260709 12:26:49.520744 30984 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:49.521636 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.475499  6027 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.530156 31661 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:49.507169  6068 raft_consensus.cc:2764] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:26:49.532651 30958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.018s	sys 0.012s
W20260709 12:26:49.533098 30958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.018s	sys 0.012s
W20260709 12:26:49.533612 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 567023us
I20260709 12:26:49.545763 31063 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:49.529907  6052 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:49.533885  6047 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:49.540033 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:49.565652  6047 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:49.589654 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 175731us
I20260709 12:26:49.553990  6023 raft_consensus.cc:515] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.594106  5831 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:49.594559  5831 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:49.594755 31662 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.594936  5831 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:49.596396  6027 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.598212 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:49.598803 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 58876.4us
I20260709 12:26:49.599212 31698 raft_consensus.cc:2410] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:26:49.590312  6047 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:49.606555 31695 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:49.608922 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:49.609707 31058 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.603417  6027 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:49.614670 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:49.622956 31058 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:49.624593 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:49.624799  6093 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:49.626073  6093 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.628549  6093 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.625175 31059 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:49.631239 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:49.631860 31068 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:49.604735 31694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:49.638852 31694 raft_consensus.cc:606] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:49.640681 31059 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 13.
W20260709 12:26:49.647794  6100 log.cc:927] Time spent T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:26:49.655503  6099 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:26:49.663445  6023 leader_election.cc:290] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.630752  6027 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.668090  6027 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.673182 31135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2918) took 1120 ms. Trace:
I20260709 12:26:49.677443  6075 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:49.683609  6050 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:49.688300 31106 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.691192 31691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1533) took 1039 ms. Trace:
I20260709 12:26:49.690800  5831 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.691685 31691 rpcz_store.cc:276] 0709 12:26:48.652232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.652494 (+   262us) service_pool.cc:224] Handling call
0709 12:26:48.652746 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.827605 (+174859us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:49.038682 (+211077us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.421075 (+382393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:49.421145 (+    70us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:49.421213 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.690901 (+269688us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b5400801bd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:49.690964 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.691119 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":476804,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":176,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":4276,"spinlock_wait_cycles":401511808,"thread_start_us":576102,"threads_started":2,"wal-append.queue_time_us":508402}
I20260709 12:26:49.695443  6047 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.697153  5852 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.699028 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:49.703560  5831 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.704511  6082 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:49.706682 31140 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42006 (request call id 1401) took 1126 ms. Trace:
I20260709 12:26:49.707108 31140 rpcz_store.cc:276] 0709 12:26:48.579828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.581057 (+  1229us) service_pool.cc:224] Handling call
0709 12:26:48.631394 (+ 50337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.631649 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:48.867822 (+236173us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.706249 (+838427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:49.706319 (+    70us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:49.706401 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.706487 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.706614 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":572526,"prepare.queue_time_us":105094,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2956,"thread_start_us":496267,"threads_started":2,"wal-append.queue_time_us":511861}
I20260709 12:26:49.709723 31136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2916) took 1206 ms. Trace:
I20260709 12:26:49.710090 31136 rpcz_store.cc:276] 0709 12:26:48.503496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.513502 (+ 10006us) service_pool.cc:224] Handling call
0709 12:26:48.513764 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.536919 (+ 23155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.536949 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.574657 (+ 37708us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.144178 (+569521us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:49.252881 (+108703us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.484970 (+232089us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:49.709322 (+224352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.709402 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.709443 (+    41us) raft_consensus.cc:1685] finished
0709 12:26:49.709541 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.709670 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":497713,"prepare.queue_time_us":108153,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":24025,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":10108,"spinlock_wait_cycles":10879232,"thread_start_us":536151,"threads_started":1,"wal-append.queue_time_us":712080}
I20260709 12:26:49.712551 31664 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:49.712864  5775 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:49.712601 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:49.715118  6040 raft_consensus.cc:2804] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:49.716809  5934 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:49.717165  6020 raft_consensus.cc:993] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:49.717547 31109 leader_election.cc:304] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:49.718730 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
W20260709 12:26:49.719018  6103 log.cc:927] Time spent T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:49.719332  5389 raft_consensus.cc:2749] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:49.719785  6052 raft_consensus.cc:1081] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:49.720031 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.720171  5776 raft_consensus.cc:2764] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:26:49.721537  6040 raft_consensus.cc:697] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:49.722512  6040 consensus_queue.cc:237] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.724869 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.725446 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.726063 31138 raft_consensus.cc:1275] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:49.726311 31139 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 12:26:49.727011 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.727447 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.728574 31141 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:26:49.729214 31661 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.717348  5934 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.730144 31140 raft_consensus.cc:1240] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:49.730124  6016 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:49.730087 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:49.731875  5934 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:49.732194 31068 raft_consensus.cc:2393] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 13: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:49.732409 31136 raft_consensus.cc:1240] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:49.734174 31109 leader_election.cc:304] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.734432 31689 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:49.734933 31689 raft_consensus.cc:606] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:49.735239 31689 raft_consensus.cc:614] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:49.735525  6016 raft_consensus.cc:2749] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:49.735643 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.737342 31033 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:49.738055  6027 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:49.738552 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.740159 31137 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:49.741907 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:49.743578  5991 consensus_queue.cc:1048] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:49.744035  6000 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:49.743037  6027 consensus_queue.cc:1059] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:49.745622  5991 raft_consensus.cc:3055] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:49.745970  5991 raft_consensus.cc:740] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:49.747056  5999 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:49.746852  5991 consensus_queue.cc:260] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.748180  5991 raft_consensus.cc:3060] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.749953  5700 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:49.751283 31136 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 19. (term mismatch)
I20260709 12:26:49.751634 31136 pending_rounds.cc:85] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 18
I20260709 12:26:49.751842 31136 pending_rounds.cc:107] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 23.19
I20260709 12:26:49.752074 31136 raft_consensus.cc:2889] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:49.754565  5974 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:49.755199  5974 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:49.755735  6000 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:49.756052  5974 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.759299 30955 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:49.759831  5916 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:49.760131 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:49.760950 30955 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:49.762665 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.673703 31135 rpcz_store.cc:276] 0709 12:26:48.552338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.552523 (+   185us) service_pool.cc:224] Handling call
0709 12:26:48.576138 (+ 23615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.576334 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:48.877772 (+301438us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.052169 (+174397us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.658186 (+606017us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b180005c200. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee87a73f3 00007f2ee87a97c1 00007f2ee87a9779 00007f2ee87a9709 00007f2ee87a96d1 00007f2ee87a89cd 0000000000388634 0000000000388479 00007f2ee878a83a
0709 12:26:49.672490 (+ 14304us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b180005c200. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef166fab1 00007f2ef167c281 00007f2ee8721c0a 00007f2ef1e61219 00007f2ee87a9a40 00007f2ee87a44e4 00007f2ee87a4455 00007f2eeb88c060 00007f2eeb89e716 00007f2eeb8af287 00007f2eeb8af208
0709 12:26:49.672694 (+   204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:49.672764 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:49.672849 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.672955 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.673107 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":567790,"prepare.queue_time_us":98621,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":6341,"raft.queue_time_us":640447,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":20512,"spinlock_wait_cycles":333229696,"thread_start_us":432325,"threads_started":2,"wal-append.queue_time_us":486643}
I20260709 12:26:49.750015 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.779382 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:49.697983  6047 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:49.797991 31109 leader_election.cc:304] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:49.806295 31033 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:49.808830 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
W20260709 12:26:49.812386 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 211661us
I20260709 12:26:49.813757 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:49.697844  5852 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.815876 31033 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:49.819067 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:49.821470 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.822934 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76fe9dfd8b4c4785a354982ad7e1faa4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:49.828029 30988 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.823820 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:49.832360 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:49.833396 31691 raft_consensus.cc:2393] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 13: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:49.834729 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:49.838737 31033 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:49.839102 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:49.839983 31138 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.843374 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:49.846112  6027 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:49.780853  5874 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:49.780953  5951 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:49.783396  6075 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:49.814468  5879 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:49.886374  5879 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.863171  5590 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:49.851265  5916 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:26:49.866490  6027 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:49.867715 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.836741 31140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:42006
I20260709 12:26:49.868481 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:49.869599  5999 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.883975 31697 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 12:26:49.888195 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 53411.2us
W20260709 12:26:49.889509 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 65782.2us
I20260709 12:26:49.898180 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:49.784873  6106 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:49.891736 31068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60b6187f32774daba89763099dabeb2a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:49.813400  6000 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:49.893612 31140 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:49.891156  5775 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:49.889249  6119 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:26:49.858286  5665 raft_consensus.cc:2749] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:49.902597 31140 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:49.886783  5879 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:49.859035  5874 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:49.910008 31058 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:49.911075  5776 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:49.913108  6106 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:49.913733  6062 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:49.914247  6106 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.895573  5590 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.920542  5590 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.910674  5874 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:49.902141  5999 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.923516  5999 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.927289 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:49.929742  5852 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:49.930419 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.936210  6058 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:49.939656 31064 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 12:26:49.940053 31064 pending_rounds.cc:85] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 15
I20260709 12:26:49.940266 31064 pending_rounds.cc:107] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:26:49.940488 31064 raft_consensus.cc:2889] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:49.943547 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:49.898248  5951 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:49.943876  5874 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.944384 31059 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:49.948381 30988 raft_consensus.cc:2468] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:26:49.897171  5967 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:49.946902 31140 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.901221  6027 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:49.956873 31664 leader_election.cc:304] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:49.962388  6027 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.955350  5999 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:49.965930  6101 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:49.966045  5999 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.966522  6101 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.958660  5967 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.968160  5999 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.968758  6101 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.944866  5951 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:49.970369  5572 raft_consensus.cc:2804] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:49.970845 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:49.971083 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:49.968838  5967 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:49.954510  5669 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:26:49.970894  5572 raft_consensus.cc:697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:49.983098  5572 consensus_queue.cc:237] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:49.984309 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.985455 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:49.986958 31140 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:49.986022 31142 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:50.002956 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:50.008152 31693 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 17. Preceding OpId from leader: term: 18 index: 17. (term mismatch)
I20260709 12:26:50.008680 31693 pending_rounds.cc:85] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 16
I20260709 12:26:50.008982 31693 pending_rounds.cc:107] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 17.17
I20260709 12:26:50.009274 31693 raft_consensus.cc:2889] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:49.974323  5949 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:49.976898  6094 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:50.024060 31138 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:50.033983 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.035710 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.036854 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.037190 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.038431  5874 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.038833  6094 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.088618 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.088670 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:50.019973  5916 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:50.046695  5949 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.129914 30990 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:50.126890  5991 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.157405 31138 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:50.159610 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 122598us
I20260709 12:26:50.160969 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:50.161986  6094 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.162949  5879 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.165107  5879 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:50.170751 31139 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 23: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:50.159888  6034 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:50.171729  6034 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.173787  6034 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:50.152029  5991 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.175383  5949 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:50.152719  6062 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:50.176594  5991 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.176684  6062 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.177574 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:50.180042 30958 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:50.181183  6094 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:50.181660 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:50.183720 30992 raft_consensus.cc:2468] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:50.185823 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:50.194598 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.196568 31691 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:50.178157 31144 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:50.202088 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:26:50.213019  6115 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:26:50.178941  6062 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:50.235591 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:50.239763 31060 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:50.236047 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:50.242668 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 341714us
W20260709 12:26:50.246034 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 340822us
I20260709 12:26:50.244966 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:50.236231 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 52079.6us
I20260709 12:26:50.146203  5916 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.238574  5977 raft_consensus.cc:493] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:50.265127 30992 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:50.266259 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.287614 31068 raft_consensus.cc:606] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Received request to transfer leadership to TS e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:50.244812  6062 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:50.290165 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.296424 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.297230 30985 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:50.299726 31063 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:26:50.312496 31109 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:50.323218 30989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:50.271246  5776 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:50.245163  5389 consensus_queue.cc:1059] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 12:26:50.287696  6034 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.331235 30989 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 23: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:50.334760 31060 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 24.
I20260709 12:26:50.294250  6062 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.337852 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:50.341473 30992 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.340592 31064 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:26:50.343223 30955 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:50.335457  5999 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:50.292546  5977 raft_consensus.cc:515] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.356053 31063 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:26:50.353609  5775 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:50.343302  6062 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.359161  5572 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:50.363171  5831 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:50.363713  5831 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:50.364526  5831 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.307502  6027 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:26:50.373171 30992 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:50.368198  5389 raft_consensus.cc:3055] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:50.424727 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48096us
I20260709 12:26:50.339927  5776 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.344643  6034 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:50.422875  6106 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.005s	sys 0.023s
I20260709 12:26:50.374998  6011 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
W20260709 12:26:50.440394 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 299826us
I20260709 12:26:50.428546  5389 raft_consensus.cc:740] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:50.443387 31138 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:26:50.437142  6106 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.005s	sys 0.023s
W20260709 12:26:50.475512 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 272352us
I20260709 12:26:50.476251 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:50.427021  5879 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:26:50.429320  5852 consensus_queue.cc:1048] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:26:50.485808 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:50.508013 31694 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:50.508350 31694 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:50.486197 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:50.509836 31691 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:50.512508 31696 raft_consensus.cc:1275] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:26:50.517989 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:50.522316 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:50.516328 31694 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.524591 31694 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:50.532820 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.536695  5837 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.537236  5837 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.540858 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.440124  5934 consensus_queue.cc:1048] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:26:50.512181  6050 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:50.511202  5879 consensus_queue.cc:1243] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:26:50.476832  6051 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:26:50.519979  6052 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
W20260709 12:26:50.568457 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 317068us
I20260709 12:26:50.568838  5837 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:50.571681 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 128941us
I20260709 12:26:50.584527  5389 consensus_queue.cc:260] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.547322  5916 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:50.559455  6122 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:26:50.596558  5389 raft_consensus.cc:3060] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.596714 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:50.603643 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 173351us
I20260709 12:26:50.608971 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:50.565728 31694 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 21.
I20260709 12:26:50.650482 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.702857 31062 raft_consensus.cc:2393] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 25: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:50.553951  6050 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.605871  5932 raft_consensus.cc:493] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.630664 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
W20260709 12:26:50.655965  6133 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 12:26:50.712756 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 82185.3us
W20260709 12:26:50.657475  6128 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 12:26:50.541680  6020 consensus_queue.cc:1175] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce27a72886964e4287059a1d66bd5b6b" responder_term: 21 status { last_received { term: 20 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:26:50.658937  6125 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
W20260709 12:26:50.659760  6126 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:50.663000  6034 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.663940 31060 pending_rounds.cc:85] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 16
I20260709 12:26:50.786003 31060 pending_rounds.cc:107] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:26:50.786296 31060 raft_consensus.cc:2889] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:50.664386  6098 log.cc:927] Time spent T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:26:50.665071  6135 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:26:50.674832  6120 log.cc:927] Time spent T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.167s	user 0.003s	sys 0.000s
I20260709 12:26:50.628207  6124 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:50.654780  6051 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:50.629742  6123 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:50.590960  6052 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.678709  5977 leader_election.cc:290] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.680387  5776 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:50.682492  5999 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
W20260709 12:26:50.682889  6121 log.cc:927] Time spent T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:26:50.630424 31068 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:50.621281  5874 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:50.677661  6062 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.797811  6157 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:50.847258  6157 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.848188  6062 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.848964 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.701804 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.851364  6050 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.780258 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.852113 31065 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51942 (request call id 3347) took 1026 ms. Trace:
I20260709 12:26:50.852473 31065 rpcz_store.cc:276] 0709 12:26:49.825657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.825840 (+   183us) service_pool.cc:224] Handling call
0709 12:26:49.826209 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.826669 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.826710 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.827252 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.146920 (+319668us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:50.146965 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.147030 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.851936 (+704906us) raft_consensus.cc:1685] finished
0709 12:26:50.851955 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.852064 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1043,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":20530,"thread_start_us":319313,"threads_started":1,"wal-append.queue_time_us":518065}
I20260709 12:26:50.780902  5934 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:50.853703 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.854241  5934 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:50.855194  5934 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.838101  5776 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:50.858439  5776 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.839545  6101 raft_consensus.cc:493] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:50.861047  6101 raft_consensus.cc:515] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.863058  6101 leader_election.cc:290] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.840080  5831 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:50.863703  5831 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.865785  5831 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:50.835172 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 131631us
I20260709 12:26:50.869331 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.870036 30958 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:50.788280  6144 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.789044  5998 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.700897  5775 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:26:50.879455 31137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 3015) took 1117 ms. Trace:
I20260709 12:26:50.870723 31141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45360 (request call id 3013) took 1116 ms. Trace:
I20260709 12:26:50.882103  5590 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:50.875808 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.876205 31067 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:50.795406  6124 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.721427  5932 raft_consensus.cc:515] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:50.901751  5932 leader_election.cc:290] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.902719 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:50.879804 31137 rpcz_store.cc:276] 0709 12:26:49.762532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.762671 (+   139us) service_pool.cc:224] Handling call
0709 12:26:49.808802 (+ 46131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.812957 (+  4155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.813003 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.813498 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.152937 (+339439us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:50.152979 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.153042 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.674323 (+521281us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b4400022b4c. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ee862fdb1 00007f2ee86cdc85 00007f2ee86ce6a2 00007f2eeb4401db 00007f2eeb43baaf 00007f2eeb42f86e 00007f2eeb42ebb6 00007f2eeb42e7d8 00007f2eeb438221 00007f2eeb4381d9 00007f2eeb438169
0709 12:26:50.870146 (+195823us) raft_consensus.cc:1685] finished
0709 12:26:50.870166 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.879390 (+  9224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7874,"raft.queue_time_us":13417,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":21289,"spinlock_wait_cycles":10462976,"thread_start_us":339154,"threads_started":1,"wal-append.queue_time_us":625444}
I20260709 12:26:50.887423 31063 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:26:50.880054 31141 rpcz_store.cc:276] 0709 12:26:49.754134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.754258 (+   124us) service_pool.cc:224] Handling call
0709 12:26:49.754538 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.760777 (+  6239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.760813 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.761327 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.133866 (+372539us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:50.134475 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.134533 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.870475 (+735942us) raft_consensus.cc:1685] finished
0709 12:26:50.870502 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.870657 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5861,"raft.queue_time_us":6952,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":372113,"threads_started":1,"wal-append.queue_time_us":669534}
I20260709 12:26:50.875551  5831 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.907894  5831 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:50.914808 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 429288us
I20260709 12:26:50.915683 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:50.917109 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd024d82eb9c473ba31252aeeb287fc0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:50.890239 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:50.893801 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:50.919556 31694 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:50.899487  6124 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.894641 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:50.844762  5999 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:50.931807 30984 raft_consensus.cc:1240] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:26:50.849293  6157 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:50.875931  6160 log.cc:927] Time spent T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:26:50.885886  6027 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:50.943759 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.944622 30992 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 20.
I20260709 12:26:50.923089 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.935341 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.935405 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.946820 31059 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 25.
W20260709 12:26:50.922400 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 446210us
I20260709 12:26:50.937723 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:50.936291 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.949999 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.950848 31033 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:50.943755 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:50.950758 30991 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 25.
I20260709 12:26:50.982729 31138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:50.928164  5831 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:50.988117 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47236.5us
I20260709 12:26:51.015617 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:51.020344  6052 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:50.879242  6106 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:26:51.021731 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.023432 31139 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:50.980918 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:50.984138 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 46312.3us
I20260709 12:26:50.935473  6011 raft_consensus.cc:3055] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:50.844476  6123 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:50.985504  5949 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:51.028899 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95980265645c4f7a8d6b201f0ae58174" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:50.987820  6141 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:50.938830  5932 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:51.012662  6027 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:51.013445  5874 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:51.013808  6093 raft_consensus.cc:493] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:51.118479 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:50.915062  5998 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:51.027558 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 46766us
I20260709 12:26:51.122102 31033 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:51.032380 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 88736.5us
I20260709 12:26:51.050666 31698 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:51.028424  5967 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:51.053207 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.054791 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd1ada4571045f6ba305e207cccddcf" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:51.056675  6132 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:26:51.059259  6123 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.026437  6011 raft_consensus.cc:740] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:51.061474 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.154904 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:50.884034  6144 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:51.157271 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 95867.4us
I20260709 12:26:51.158279 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.159665 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:51.062335  5669 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:51.062892 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.068157  6058 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:51.068826  6062 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:51.082989 31135 raft_consensus.cc:1275] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:26:51.096324 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 75040.1us
I20260709 12:26:51.097039 31664 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.097023 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.123664 31143 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:26:51.130591 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.163633 31143 pending_rounds.cc:85] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Aborting all ops after (but not including) 13
I20260709 12:26:51.144827 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:51.149469 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.163887 31143 pending_rounds.cc:107] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:26:51.151373  5998 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
I20260709 12:26:51.116676  5977 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:51.119135  5932 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.116353  6027 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:51.152388  5991 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:51.118736  5874 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:51.201274 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.154966 31693 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:51.162448 31137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.163061 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ec17bf7d52c4cb69f4d488858fecc8b" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.150169 30958 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.164197 31143 raft_consensus.cc:2889] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:51.123847  5967 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.169704 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.170543 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.142813  5837 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:51.199656 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 102711us
I20260709 12:26:51.169415  6047 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:51.207588 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57499.4us
I20260709 12:26:51.169886  6051 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:51.210829 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.212916 31664 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.124583  6093 raft_consensus.cc:515] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.169286  6058 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:51.214406 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.222632 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.194927  6141 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:51.144670  5999 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.207890  5967 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:51.195195  6152 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:51.195168  6082 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.219722 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:51.171379  6123 consensus_queue.cc:1048] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:26:51.209841  5152 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:51.209460  6020 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.249370 30985 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:51.238899 30986 raft_consensus.cc:1240] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:51.218292  6154 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:51.216321  6051 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:51.227880  6106 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.252924 30987 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.254236 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.228767  6141 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:51.224897  6138 raft_consensus.cc:2749] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:51.194829  5977 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.265543 31059 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:51.266040 31142 raft_consensus.cc:1240] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:51.226809  6058 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:51.217298  6047 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.268319 30989 raft_consensus.cc:1275] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:51.291852 30988 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.275154 31664 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.293601 30988 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 20.
I20260709 12:26:51.262440 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.295768 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:51.214238  5837 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.230459  6101 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:51.298434 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.313575  5839 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:51.314822 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.315626  5669 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.315945 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.316725  5669 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.274685  6051 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:51.314707  6138 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:26:51.321475  6167 log.cc:927] Time spent T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:26:51.323077 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 152638us
W20260709 12:26:51.307283 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 76620.3us
I20260709 12:26:51.323833 31033 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:51.291793  6141 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.226531  6011 consensus_queue.cc:260] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.332831  6062 raft_consensus.cc:2804] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:51.300042  6020 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:51.302045  6082 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:51.267556  6149 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:26:51.311743  5849 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:51.345705 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.334450 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.297685  6023 consensus_queue.cc:1048] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:26:51.331348  6154 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:51.350759 30958 leader_election.cc:304] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.345822  5849 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:51.354005  5849 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:51.358593  6152 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:51.359026  6144 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.361609  6154 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.363114 31068 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 16.
I20260709 12:26:51.365641 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.366557 30958 leader_election.cc:304] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.367648 30958 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:51.364182  6165 log.cc:927] Time spent T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:26:51.377732  6081 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:51.387161  6047 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:51.387995  6093 leader_election.cc:290] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.391353  5999 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.393640  5776 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:51.395963  5977 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.397291  5837 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:51.397179  6124 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.364910  6171 raft_consensus.cc:2749] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:51.442493  6027 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.444306  6124 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:51.444542  6027 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.439850  6058 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.445576  5874 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.446758  6058 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.373416  6101 raft_consensus.cc:2749] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:51.447886  5874 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.451786 31697 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:51.399873  5932 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.453186 31138 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 21: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:51.400007 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.401548 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.347059  6062 raft_consensus.cc:697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:51.411659 30955 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:51.413862 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:51.468050 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 56514.2us
I20260709 12:26:51.373342  5892 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:51.422212 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:51.427860 30958 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.438422  6154 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.457293 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:51.484345 31067 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:51.470570 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:51.485351 31062 raft_consensus.cc:2393] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 18: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
W20260709 12:26:51.485194 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 57418.6us
I20260709 12:26:51.471588 30991 raft_consensus.cc:3060] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.445899  5967 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.491127 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:51.483780  5700 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:51.501498 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 154095us
I20260709 12:26:51.489339  6101 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.495554  6023 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.499797  5700 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:51.489466  5879 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:26:51.501050  6062 consensus_queue.cc:237] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.511250  6051 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.514801  6051 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.516607 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:51.506513  5776 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:51.510773  5700 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:51.502802  5831 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:26:51.544807  6124 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:51.546195 30991 raft_consensus.cc:2468] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 21.
I20260709 12:26:51.560212 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.561887 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:51.508401  6023 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.562958  6124 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.568743  5837 consensus_queue.cc:1059] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:51.611755  5837 raft_consensus.cc:3055] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:51.612128  5837 raft_consensus.cc:740] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:51.613041  5837 consensus_queue.cc:260] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.569445  6020 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:51.571624  6081 consensus_queue.cc:1059] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:51.571915  6156 consensus_queue.cc:1048] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:51.615667  6023 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:51.596529  6141 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.617708  6124 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:51.599699  6166 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.054s	user 0.000s	sys 0.002s
I20260709 12:26:51.618435  5837 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:51.619203  6141 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.623472 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:51.575518  6047 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:51.575668 31664 leader_election.cc:304] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.577745  5967 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.584949  6040 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:51.547983  5776 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.589591 31693 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:26:51.590185 31694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
W20260709 12:26:51.634567 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 68620.8us
I20260709 12:26:51.591145 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:51.637030 31063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.637499 31064 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:51.597369 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dab2cd673924dc2a2f8622e76701947" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:51.639628 30986 raft_consensus.cc:2393] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:51.599586 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.600996 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a3e0211fca8429086a4403f1ec42a59" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.640549 30985 raft_consensus.cc:2410] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:26:51.602377 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.604988 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.605515 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.605589 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:51.627985 31060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.628432 31144 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22. Ops: [21.16-21.16]
W20260709 12:26:51.633620 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 58027.2us
I20260709 12:26:51.653609 31063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:51.654055 31033 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:51.654358 31063 raft_consensus.cc:606] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.654768 31063 raft_consensus.cc:614] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:51.655432 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:51.656539 31664 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.569824  5849 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.630445  6047 raft_consensus.cc:493] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.658324 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:51.660595 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:51.673986 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.630813  6155 raft_consensus.cc:493] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.675212 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:51.631044  6153 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:51.625990  5700 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.648306  6023 raft_consensus.cc:2749] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.689704 30992 raft_consensus.cc:1240] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 25. Current term is 26. Ops: []
W20260709 12:26:51.683478 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 48552.4us
I20260709 12:26:51.691085 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.692538 31664 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.695066 30985 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:51.662022  5669 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:51.725306 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "563f5eec53c14273959c9883d2423cc9" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.670166  6081 consensus_queue.cc:1243] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Peer e9cccf7eb2a545ef8c2169b67ff261bf log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:51.669431 31694 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 18.
I20260709 12:26:51.652973  6040 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.758805 31063 raft_consensus.cc:1240] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:51.759269 31067 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:51.718926  6144 raft_consensus.cc:2804] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:51.714496  5951 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:51.766000 31064 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 26.
W20260709 12:26:51.742847  6188 log.cc:927] Time spent T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:26:51.688717  6155 raft_consensus.cc:515] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.745672 31697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:51.746963  5849 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:51.747352  5776 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.750026  5991 consensus_queue.cc:1059] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:51.721975 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:51.670742  6047 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.707772 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:51.811642 30991 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:51.760625 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:51.684263  5831 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:51.767473 30955 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:51.773800 30958 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.682682  6192 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.102s
W20260709 12:26:51.813025 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 45669.5us
I20260709 12:26:51.815161 31058 raft_consensus.cc:2368] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 25. Current term is 26.
W20260709 12:26:51.818895 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 45204.4us
I20260709 12:26:51.823691 30958 leader_election.cc:304] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:51.716151  6153 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.827693 31106 leader_election.cc:400] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 25. Current term is 26.
I20260709 12:26:51.828863 31697 raft_consensus.cc:606] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:51.830263 31106 leader_election.cc:403] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:26:51.830360 31697 raft_consensus.cc:614] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:51.814273  6144 raft_consensus.cc:697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:51.810901  6081 raft_consensus.cc:493] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:51.817955  5932 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:51.845992 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:51.825207  6105 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:51.854187  6198 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:51.833758  5974 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:51.813938  6016 consensus_queue.cc:1059] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:51.837102  6058 raft_consensus.cc:2749] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:51.818120  5879 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:26:51.822585  5831 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.833323  5999 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:26:51.857740  5977 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:51.849148  6149 consensus_queue.cc:1059] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:26:51.861141  5932 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:51.880517 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
W20260709 12:26:51.884862 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 52538.1us
I20260709 12:26:51.886291 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.895996  6155 leader_election.cc:290] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.900393 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.860162  6081 raft_consensus.cc:515] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:51.923887 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:51.925110 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:51.928237 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:51.933768 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:51.896062  5932 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.941354  6153 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:51.932654  6196 log.cc:927] Time spent T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 12:26:51.937438  6191 log.cc:927] Time spent T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:26:51.954404  6040 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.956813  6081 leader_election.cc:290] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:51.960688  5831 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:51.905164  6144 consensus_queue.cc:237] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:51.966472 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:51.966444 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:52.021709 31663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.032519  6198 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 25. Current term is 26.
I20260709 12:26:52.052134  6047 raft_consensus.cc:515] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.065603 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:52.072778  6047 leader_election.cc:290] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803)
W20260709 12:26:52.091028  6201 log.cc:927] Time spent T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:52.240810  6058 raft_consensus.cc:3055] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:52.241415  6058 raft_consensus.cc:740] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:52.248939  6011 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:52.245826  6058 consensus_queue.cc:260] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.262812  6058 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:52.273030  5849 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:52.273586  5849 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:52.274605  5849 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:52.276019  5849 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.284008 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60b6187f32774daba89763099dabeb2a" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:52.284678 30986 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.293049  5998 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:52.293674  5998 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.296376  5998 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:52.302997 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:52.303930 31058 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:52.307886 31059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38956 (request call id 1954) took 1238 ms. Trace:
I20260709 12:26:52.308398 31059 rpcz_store.cc:276] 0709 12:26:51.069393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.071564 (+  2171us) service_pool.cc:224] Handling call
0709 12:26:51.071845 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.299084 (+227239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.307603 (+1008519us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54002cddd4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:52.307809 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444992896}
I20260709 12:26:52.310269  6023 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:52.311098  6023 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:52.311429  6023 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:52.312384  6023 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.313645  6023 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:52.331323 30986 raft_consensus.cc:2468] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 21.
I20260709 12:26:52.338922 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:52.348485 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:52.348512 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.350044 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.350381 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
W20260709 12:26:52.354395 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 229019us
I20260709 12:26:52.358269 31664 leader_election.cc:304] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:52.363528 30985 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:52.365413 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:52.371359 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.387701 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:52.394238 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
W20260709 12:26:52.395025 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 44785us
I20260709 12:26:52.395991 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:52.368575  6040 raft_consensus.cc:2804] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:52.401347 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:52.404275 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:52.413544 31691 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:52.407763  6192 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:52.425297 31064 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:52.417708  6154 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:52.430567 30985 raft_consensus.cc:1240] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:52.396225  5932 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.407461  5991 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:52.388867  6144 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:52.440539  5932 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.444803  6144 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.446776  5932 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:52.447002  6144 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:52.452507 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 48341.3us
I20260709 12:26:52.427527  5775 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:52.422348  6040 raft_consensus.cc:697] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 21 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:52.429721  6075 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:52.483600 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:52.477142  6211 log.cc:927] Time spent T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:26:52.489233  6203 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.489780  6203 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.491977  6203 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:52.494539 31059 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:52.499259  6081 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:52.499960  6194 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:52.500357  6194 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:52.500427 31109 leader_election.cc:304] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:52.501089  5849 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:52.501188  6194 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:52.501948  5916 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:52.503343  6153 raft_consensus.cc:2749] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:52.441237  6209 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.044s
W20260709 12:26:52.524973 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 71509.5us
I20260709 12:26:52.511848  6194 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:52.529181 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:52.530689 31139 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:52.534135 31059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:52.532135 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:52.537901 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:52.538079 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:52.541579 31033 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:52.540926  6206 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:52.544737  6192 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:52.545737  6206 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.544567  6202 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:52.546437  6206 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:52.536499  6040 consensus_queue.cc:237] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.546737  6202 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.548687  6202 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:52.551985  6075 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:52.552551  6075 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:52.553793  6192 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.554283  6192 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:52.541217  5999 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:26:52.556324  6192 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:52.529882  6081 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:52.557483  6081 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.559511  6081 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:52.536362  5991 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:52.563944 31143 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:52.566545 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:26:52.567535  6215 log.cc:927] Time spent T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 12:26:52.570607 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.574869  6075 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:26:52.573514 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 206316us
I20260709 12:26:52.577144 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:52.565937  6154 raft_consensus.cc:493] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:52.572105  5916 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:26:52.606395 30957 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.607604 31058 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:26:52.580896  6144 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:26:52.587128  5974 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:52.586211  6154 raft_consensus.cc:515] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:52.587735  5916 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:52.625157  6193 log.cc:927] Time spent T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.753s	user 0.000s	sys 0.009s
I20260709 12:26:52.616055  6058 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:26:52.631691 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76399531471a4790a56d52a6ea052864" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:52.632825 31136 raft_consensus.cc:1240] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:26:52.615449  5974 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:52.654904 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 154547us
W20260709 12:26:52.746120 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 88855.9us
I20260709 12:26:52.750803 31698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51362 (request call id 1564) took 1142 ms. Trace:
I20260709 12:26:52.752437 31694 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 21 LEADER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:52.764497 30986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.765228 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.751456 31698 rpcz_store.cc:276] 0709 12:26:51.608302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.608447 (+   145us) service_pool.cc:224] Handling call
0709 12:26:51.608705 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:51.629278 (+ 20573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:51.629323 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:51.642647 (+ 13324us) log.cc:844] Serialized 26 byte log entry
0709 12:26:51.782394 (+139747us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:51.828522 (+ 46128us) log.cc:844] Serialized 14 byte log entry
0709 12:26:51.828732 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.645389 (+816657us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b54005c1a54. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:52.750422 (+105033us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:52.750465 (+    43us) raft_consensus.cc:1685] finished
0709 12:26:52.750555 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.750728 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45936,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":2651,"raft.queue_time_us":5104,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":25629,"spinlock_wait_cycles":46314752,"thread_start_us":90997,"threads_started":1,"wal-append.queue_time_us":258968}
I20260709 12:26:52.769860  6010 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.765973  5852 consensus_queue.cc:1059] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:26:52.771867  6010 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:52.773053  5852 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:52.773408  5852 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:52.753618  6040 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:52.774276  5852 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:52.774787  6040 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.775655  5852 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.776818  6040 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:52.773919  6010 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:52.768250  5665 raft_consensus.cc:493] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:52.783973  6123 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:52.783892  5665 raft_consensus.cc:515] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.784435  6123 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:52.785282  6123 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.786196  5665 leader_election.cc:290] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:52.803211 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:52.797887  6123 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:52.804718  5916 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:52.807157  5974 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:52.804244 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:52.814131 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.824396 31109 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:52.803073  6081 raft_consensus.cc:493] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:52.831416 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:52.838855 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.798292  6154 leader_election.cc:290] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:52.848234 30986 raft_consensus.cc:1240] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Rejecting Update request from peer 2c4511d66b474ee0aa41c9b1f03b66ab for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:52.849222 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d521786b7ccd41da81f7fd0d175bf582" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.849330 30958 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:52.849705 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:52.850659 30992 raft_consensus.cc:1275] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:52.852267 30990 raft_consensus.cc:1275] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:52.828104  6016 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:52.860251 31109 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.790006  6206 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.841329  5949 raft_consensus.cc:493] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:52.874580 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:52.874842 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:52.880162 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:52.865212  5951 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:52.889879  6206 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:52.841158  6081 raft_consensus.cc:515] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.891647 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:52.922343 31691 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:52.923571  6081 leader_election.cc:290] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:52.872150  5949 raft_consensus.cc:515] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:52.881108 31106 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:52.927992 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3de96a6f20142c0a00ff9ee12bcc86c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:26:52.932973 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 51974.3us
I20260709 12:26:52.934897 31068 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38956 (request call id 1959) took 1536 ms. Trace:
I20260709 12:26:52.935407 31068 rpcz_store.cc:276] 0709 12:26:51.398571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.399749 (+  1178us) service_pool.cc:224] Handling call
0709 12:26:52.934662 (+1534913us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54000fc154. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb8982fa 00007f2ef0859b17 00007f2ee9dd46d4 00007f2ee9dd4662 00007f2ee9dd4591 00007f2ee9dd450c 00007f2ee9dd37c2 00007f2ee9bcf48c 00007f2ee9bce8c6 00007f2ee9bce26f
0709 12:26:52.934826 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686586624}
I20260709 12:26:52.947139 31691 raft_consensus.cc:2468] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:52.949012 31033 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:52.953078 31106 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:52.959354 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:52.958036 31661 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:26:52.961370 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 102333us
I20260709 12:26:52.934403  5951 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:52.944521  6011 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:52.955986  5999 raft_consensus.cc:2804] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:52.971478  5949 leader_election.cc:290] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:52.961828  6016 raft_consensus.cc:2749] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:52.973498  5977 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:52.984258 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:52.993705  5951 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.998848  5999 raft_consensus.cc:697] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:52.999966  5999 consensus_queue.cc:237] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.954533  6206 consensus_queue.cc:1059] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:26:53.001564  6223 raft_consensus.cc:493] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:53.002205  6206 consensus_queue.cc:1243] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Peer a517aadb0ae04b89a13779d7898dfa0c log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:53.002722  6178 raft_consensus.cc:3055] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:53.003118  6178 raft_consensus.cc:740] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:53.002317  6223 raft_consensus.cc:515] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.003883  6178 consensus_queue.cc:260] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:52.967628  5590 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:53.014480  6178 raft_consensus.cc:3060] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.023164 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
W20260709 12:26:53.032101 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 169200us
I20260709 12:26:53.020901  6223 leader_election.cc:290] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.047400 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d43e79f1b224452ad15ff525d3a0bd9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.043622 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.018699  5590 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:53.051209 30956 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.051713 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e81e4942d094afda44355ba1d9c7735" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:53.034394  5700 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.053707 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:53.062429 31135 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 17.
I20260709 12:26:53.052765  6206 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:53.073942 30958 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:53.075443  5999 consensus_queue.cc:1048] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:53.081600 31033 leader_election.cc:304] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.063426  6192 consensus_queue.cc:1048] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 12:26:53.087990  6206 raft_consensus.cc:2749] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:53.082405  5874 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:53.111065  6106 raft_consensus.cc:493] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:53.111611  6106 raft_consensus.cc:515] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.113791  6106 leader_election.cc:290] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:53.117695  5669 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:53.118251  5669 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.120260  5669 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:53.126834 31664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.015s	sys 0.027s
W20260709 12:26:53.127168 31664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.016s	sys 0.027s
W20260709 12:26:53.128904 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 527181us
W20260709 12:26:53.129351  6240 log.cc:927] Time spent T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:26:53.130671 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.132102  6203 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:53.135967 31139 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45364 (request call id 2960) took 1534 ms. Trace:
I20260709 12:26:53.136334 31139 rpcz_store.cc:276] 0709 12:26:51.601607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.601767 (+   160us) service_pool.cc:224] Handling call
0709 12:26:51.602019 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.531062 (+929043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.135762 (+604700us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b54000947d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb894e01 00007f2eeb89383e 00007f2ef0858419 00007f2ee9dcfbd4 00007f2ee9dcfb62 00007f2ee9dcfa91 00007f2ee9dcfa0c 00007f2ee9dcecc2 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:53.135913 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2134951296}
I20260709 12:26:53.140194  5590 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:53.140892 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.141541 31141 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:53.141621 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.143503 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.145756 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.146384 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.156981 31058 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:26:53.157375 31058 pending_rounds.cc:85] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting all ops after (but not including) 13
I20260709 12:26:53.157605 31058 pending_rounds.cc:107] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:26:53.157822 31058 raft_consensus.cc:2889] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:53.165730  6023 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:53.172890 30958 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.173275 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.173951 31695 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:53.175904 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.178123 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.178182 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d50d1465e4b84136bf90de700a93ef2c" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.182731  6105 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:53.184670 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.185896 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.184054 30880 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 23, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:53.186503 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.188038 30880 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 19, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 19 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.189793 30880 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 19, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 19 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.190795 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.191376 30880 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:53.192718  6023 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:53.194070 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.194715 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05394269d1a4461bb79d6f93ab25ff4e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.217882 30991 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:53.196604 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.220357 31139 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 22: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
W20260709 12:26:53.226089 30888 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 316) took 49134 ms (client timeout 15000 ms). Trace:
W20260709 12:26:53.226531 30888 rpcz_store.cc:269] 0709 12:26:04.092291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.092584 (+   293us) service_pool.cc:224] Handling call
0709 12:26:53.225972 (+49133388us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:48.908565 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:48.952045 (+ 43480us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:48.952101 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:26:48.952123 (+    22us) tablet.cc:662] Decoding operations
0709 12:26:48.952783 (+   660us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:48.952868 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:48.952908 (+    40us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:26:48.953379 (+   471us) tablet.cc:701] Row locks acquired
0709 12:26:48.953431 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:26:48.953589 (+   158us) write_op.cc:270] Start()
0709 12:26:48.953727 (+   138us) write_op.cc:276] Timestamp: P: 1783600008953522 usec, L: 0
0709 12:26:48.953765 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:26:48.982469 (+ 28704us) log.cc:844] Serialized 1684 byte log entry
0709 12:26:49.942813 (+960344us) op_driver.cc:464] REPLICATION: finished
0709 12:26:50.482359 (+539546us) write_op.cc:301] APPLY: starting
0709 12:26:50.515310 (+ 32951us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:50.699721 (+184411us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:50.699849 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:51.018538 (+318689us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:51.018820 (+   282us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:26:51.018884 (+    64us) write_op.cc:312] APPLY: finished
0709 12:26:51.019166 (+   282us) log.cc:844] Serialized 49 byte log entry
0709 12:26:51.267873 (+248707us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:51.291722 (+ 23849us) write_op.cc:454] Released schema lock
0709 12:26:51.300648 (+  8926us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:51.336292 (+ 35644us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:51.337505 (+  1213us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15800,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":494617,"apply.run_cpu_time_us":44817,"apply.run_wall_time_us":849974,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":291997,"prepare.run_cpu_time_us":38789,"prepare.run_wall_time_us":445136,"raft.queue_time_us":468306,"raft.run_cpu_time_us":11396,"raft.run_wall_time_us":243917,"replication_time_us":988833,"thread_start_us":1134709,"threads_started":5,"wal-append.queue_time_us":769156,"wal-append.run_cpu_time_us":30650,"wal-append.run_wall_time_us":2207581}]]}
I20260709 12:26:53.241400 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d522233ee4c4536b4fc944d7866c4d9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.242024 31693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.242753 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2439e7934b98446ca4e7d82381e8a499" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.244717 31691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.234810 30958 leader_election.cc:304] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.253036 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.255276 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c6a2f53447447459d88ab978179c65c" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.227381  6023 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.258195 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2c202e27d24d1baad8ddf5d9db4391" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.262996 30958 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:53.263355 30991 raft_consensus.cc:1275] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:53.270133  5590 raft_consensus.cc:2749] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:53.270761 31109 leader_election.cc:304] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:53.271886 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.275206  5669 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.275718  5669 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.277798  5669 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.281224  6075 raft_consensus.cc:2749] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:53.281880  6075 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:53.281065 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:53.285451 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.285144  6203 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:53.282800  6209 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:53.282995  6172 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:53.289094 31664 leader_election.cc:304] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.290345  6016 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.288527  6152 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:53.292969 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.282341  6075 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.299307  6058 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:53.300309  6016 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.301440  5977 raft_consensus.cc:2749] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:53.288784  6203 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:53.307960 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.310046 31109 leader_election.cc:304] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:53.317839 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:53.319419 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.332355 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.336045  6016 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:53.337543  6075 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:26:53.338107  6249 log.cc:927] Time spent T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:26:53.339742  6106 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:53.339938 30958 leader_election.cc:304] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.340215  6106 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:53.340649  6106 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:53.342245  5590 raft_consensus.cc:2749] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:53.342355  6105 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:53.315841  6203 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:53.342935  6105 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:53.344864  6105 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.331023  5849 raft_consensus.cc:2749] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:53.347223  5977 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:53.303912  6209 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:53.348686 31062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:53.348465  6209 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:53.349395 31062 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:53.350615  6023 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.316640  6152 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:53.286819 30880 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 9 to 11, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 11 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.355408 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.354867 30880 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 20 to 23, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 23 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.356002 31140 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:53.359645 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:53.360365 31068 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:53.360569 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.361126 30983 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:53.364318 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:53.364471 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.365063  6058 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:53.365370  5974 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:53.365419  6228 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:53.366460 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:53.365440  6058 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:53.367583  6226 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:53.367663  6058 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.368085  6226 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.368463  6226 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:53.356705 30880 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.377389 30880 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:53.397437 30880 catalog_manager.cc:5697] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 14, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 14 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:53.403796 30880 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 20 to 22, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 22 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.406687 31059 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:53.378791 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.413128  6203 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.416366  6106 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.422740  6202 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:53.446014 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:53.427277 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.429160 30880 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.426002  6058 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:53.440567 31695 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:53.450889 31068 raft_consensus.cc:2393] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:53.425621  5967 consensus_queue.cc:1059] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:53.457394 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.457971 30990 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:53.461740 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.462379 31689 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:53.462435 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:53.463789 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.464833 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48047b2b139146c38a821328cf7bd148" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.460841 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:53.467782 31109 leader_election.cc:304] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:53.471315 31695 raft_consensus.cc:2410] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:26:53.467084  5974 consensus_queue.cc:1048] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:26:53.480927 30880 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 15, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 15 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:53.482504 30880 catalog_manager.cc:5697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.484491 30880 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 20 to 21, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 21 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.490378  5967 consensus_queue.cc:1243] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 19.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:53.491576  5967 raft_consensus.cc:3055] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:53.491914  5967 raft_consensus.cc:740] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:53.492767  5967 consensus_queue.cc:260] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.499603 31696 raft_consensus.cc:1240] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:53.502818  6202 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:53.503345  6202 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.503753  6202 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:53.506961 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44577.3us
I20260709 12:26:53.493988  5967 raft_consensus.cc:3060] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:53.466184  6023 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:53.466945  6152 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:53.516945 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:53.474381  5916 raft_consensus.cc:2749] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:53.465997  5999 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:26:53.514313  6023 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:26:53.531405  5849 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:53.542922  5849 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (detected failure of leader a517aadb0ae04b89a13779d7898dfa0c)
I20260709 12:26:53.543327  5849 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:53.539350 30880 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.549165 30880 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 19, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 19 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.553493  6058 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.554785 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:53.568011 31068 raft_consensus.cc:2393] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9cccf7eb2a545ef8c2169b67ff261bf in current term 21: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:53.554885  6106 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:53.574525 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:53.578980 31144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:53.524394  6152 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:53.571295 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:53.532927  6203 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.582276  6058 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.594511  6058 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:53.596763  6058 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.597271  6152 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:53.605662 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.608553 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:53.609390 31068 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 26.
I20260709 12:26:53.592643  6226 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.611747 31140 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 24: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:53.612481  6226 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:53.616047 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:53.621622  6209 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:53.622020  6209 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.622370  6209 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:53.622778 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:53.625874 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:53.626439 30991 raft_consensus.cc:3060] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:53.539809  6023 raft_consensus.cc:3060] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:53.633536 30991 raft_consensus.cc:2468] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 24.
I20260709 12:26:53.635542 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:53.636102  6202 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.639969  5916 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:53.649407 30880 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:53.651172 30880 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 23, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 23 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.652889 30880 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 18, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 18 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.654721 30880 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.656718 30880 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
W20260709 12:26:53.657347 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 67854.3us
I20260709 12:26:53.658313 30880 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 13 to 15, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 15 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.660461 30880 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 12 to 14, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 14 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:53.659025 31143 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 21.
I20260709 12:26:53.669353 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:53.677978 30983 raft_consensus.cc:1275] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 12:26:53.678577 30983 pending_rounds.cc:85] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting all ops after (but not including) 15
I20260709 12:26:53.679107 30983 pending_rounds.cc:107] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 12:26:53.679495 30983 raft_consensus.cc:2889] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:53.662297 31109 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:53.654029  5849 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.707233 30958 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:53.714823 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.716895  5999 consensus_queue.cc:1048] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:53.720031 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:53.720729 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:53.705175  6124 raft_consensus.cc:2804] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:53.671695  6202 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.730430 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:53.716286  6058 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:53.720029  6230 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:53.724164  5949 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:53.742120 31140 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:26:53.729410  6124 raft_consensus.cc:697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 24 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:53.662369 30880 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:53.772246  6124 consensus_queue.cc:237] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.792686 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:53.794350 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:53.795120 31068 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 23: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:53.796060 31698 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b in current term 24: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:26:53.824426 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:53.832484  6203 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:53.839375  5849 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:53.836012 30880 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 18, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 18 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.857784 30880 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 17, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 17 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.860093 30880 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:53.863281  6209 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:53.865358  6209 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 27 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.853394  6203 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:53.866565  6203 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.888520 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:53.889324 30983 raft_consensus.cc:2393] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 24: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:53.897015 30983 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:53.897825 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:53.901377  6023 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.903486  6023 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:53.903784  6016 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:53.904565  6016 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:53.904881  6016 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:53.905501  5934 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:53.905629  6016 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:53.906270  5934 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:53.906939  6016 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:53.908596  5934 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:53.910323 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:53.910972 31068 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 26.
I20260709 12:26:53.912098 31106 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:53.912926  6172 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:53.913345  6172 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:26:53.913764  6172 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:53.861754 30880 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:53.915656 30880 catalog_manager.cc:5697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 19, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 19 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:53.846946  6058 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:53.927793 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:53.931015 30991 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 26. Current term is 27.
I20260709 12:26:53.938740 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:53.940253 31058 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:53.957679 31109 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 26. Current term is 27.
I20260709 12:26:53.960137  5999 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.996017 31058 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 27.
W20260709 12:26:54.013270 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 182557us
I20260709 12:26:54.019443 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:54.019680 30991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:54.020192 31135 raft_consensus.cc:2393] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 22: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:26:54.024814 31143 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:53.983479  5999 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:54.043272 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:54.043905 31135 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:54.026376 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:54.013290  6058 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:54.045801  6228 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:54.050549 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:54.050073 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:54.054762  6178 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:54.027567 30991 raft_consensus.cc:2468] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:54.030184  5951 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:54.061517 31696 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:54.026158  6267 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:54.059562  5999 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:54.061172 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:54.055148  6178 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.072448  6178 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.037308  6172 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:54.084040  6267 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:54.092676  6267 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:54.094642 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:54.098408 30983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:54.103302  6178 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:54.104663 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:54.105454 31068 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 26. Current term is 27.
I20260709 12:26:54.104310  6172 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:54.106691 31106 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 election: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 26. Current term is 27.
I20260709 12:26:54.108626 31106 leader_election.cc:403] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:26:54.106652 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:54.109309  6172 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:54.110383  5999 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:54.110613 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:54.111290 31143 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.117262 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:54.117987 30985 raft_consensus.cc:2368] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 26. Current term is 27.
I20260709 12:26:54.118413  6178 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:54.120033 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:54.120658 31109 leader_election.cc:400] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 election: Vote denied by peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 26. Current term is 27.
I20260709 12:26:54.127754 31143 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:26:54.132511 31033 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:54.135319 31109 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:54.142089  6152 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:54.153666  6152 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:26:54.155009 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 125247us
I20260709 12:26:54.156751 30955 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:54.154901  6152 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:54.162639 31664 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:54.241410  6058 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:54.247237  5998 raft_consensus.cc:2804] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:54.164738  6172 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157) with higher term. Message: Invalid argument: T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for earlier term 26. Current term is 27.
W20260709 12:26:54.241461 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 78758.4us
W20260709 12:26:54.253247  6276 log.cc:927] Time spent T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
I20260709 12:26:54.138655  6178 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:26:54.258324 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 101512us
I20260709 12:26:54.262252  6178 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:54.264722  6178 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:54.272783  6058 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:54.276106  6058 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:54.272361  5998 raft_consensus.cc:697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:54.288405 31062 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:54.294827 30991 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:54.313025  6075 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:54.329931 31062 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:54.369603 31696 raft_consensus.cc:1275] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:26:54.370350 31696 pending_rounds.cc:85] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 15
I20260709 12:26:54.372058 31696 pending_rounds.cc:107] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:26:54.373209 31696 raft_consensus.cc:2889] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:54.408505  5998 consensus_queue.cc:237] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:54.412988 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:54.414386  5152 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:26:54.450774  5916 consensus_queue.cc:1048] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:54.482797 31058 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 23. Current term is 24. Ops: []
W20260709 12:26:54.523003  6277 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:26:54.589629  6280 log.cc:927] Time spent T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.117s	user 0.003s	sys 0.000s
I20260709 12:26:54.590822 31143 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:26:54.597394  6281 log.cc:927] Time spent T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:26:54.598953  6229 consensus_queue.cc:1059] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:54.612990  6229 consensus_queue.cc:1243] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer 2c4511d66b474ee0aa41c9b1f03b66ab log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:54.613581  6229 raft_consensus.cc:3055] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:54.613894  6229 raft_consensus.cc:740] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:54.615252  6229 consensus_queue.cc:260] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:54.616644  6229 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.600716 31135 raft_consensus.cc:1240] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:54.622673 31058 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:26:54.643915 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57559.3us
I20260709 12:26:54.658115  6106 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:26:54.671788  5932 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:54.673362  6124 raft_consensus.cc:493] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.673998  6124 raft_consensus.cc:515] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:54.676311  6124 leader_election.cc:290] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:54.679768  6267 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.693209 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:54.694571 30990 raft_consensus.cc:2410] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce27a72886964e4287059a1d66bd5b6b for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:26:54.694260  6267 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:54.696736  6267 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:54.735002 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "044e61a321dd4521be7f9ee796ca44dc" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:54.763058 31060 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:54.779690 31109 leader_election.cc:304] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:54.780838  6152 raft_consensus.cc:2749] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:54.791632  5932 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:26:54.807723 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 52541.4us
I20260709 12:26:54.819880 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:54.830852 31696 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 23.
I20260709 12:26:54.832119 31695 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:26:54.845060 31135 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:54.847409  6284 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:54.853472 31060 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:26:54.855448  6124 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:54.861056 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:54.863003 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:54.873100 31143 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:54.876777  5152 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:54.890285  5916 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:54.941452  6284 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:54.946585 30985 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:26:54.951722 31696 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:26:54.952103 31696 pending_rounds.cc:85] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 15
I20260709 12:26:54.952330 31696 pending_rounds.cc:107] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:26:54.952545 31696 raft_consensus.cc:2889] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:54.955634  6284 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:54.968015  5999 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:54.966001  6284 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.982426 31141 raft_consensus.cc:1275] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:26:54.996978  6228 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:26:55.016352  5974 consensus_queue.cc:1048] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:55.066377  6284 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.069326 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:55.070441 31058 raft_consensus.cc:1240] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Rejecting Update request from peer ce27a72886964e4287059a1d66bd5b6b for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:55.090730  5916 consensus_queue.cc:1059] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:55.091761  5916 raft_consensus.cc:3055] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:55.092109  5916 raft_consensus.cc:740] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:55.092864  5916 consensus_queue.cc:260] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.094115  5916 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:55.122641  6284 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:55.123095 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:55.125120 31033 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:55.126336  6284 raft_consensus.cc:2749] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:55.291174  6291 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:55.291815  6291 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.294195  6291 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:55.299384 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" is_pre_election: true
I20260709 12:26:55.300359 30985 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:55.303274 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:55.304004 31143 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:55.306169 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:55.308735  6228 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:55.309152  6228 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:55.309572  6228 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.338702 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0afdb545bb8f485993259007a00c1841"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:55.339538 31695 raft_consensus.cc:606] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:55.339854 31695 raft_consensus.cc:614] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:55.351817 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41116db50e8c4b9598abe43f5d36bf12"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:55.352345 31695 raft_consensus.cc:606] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:55.352698 31695 raft_consensus.cc:614] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:55.355387 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:55.355913 31695 raft_consensus.cc:606] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:55.356230 31695 raft_consensus.cc:614] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:55.358984 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:55.359505 31695 raft_consensus.cc:606] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:55.371323 31058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:60354
I20260709 12:26:55.372195 31058 raft_consensus.cc:606] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:55.373476 31058 raft_consensus.cc:614] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:55.376540 30985 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:55.377097 30985 raft_consensus.cc:606] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:55.378592 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:55.382764  6290 raft_consensus.cc:993] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:55.383320  6229 raft_consensus.cc:1081] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:55.386415 31058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:38956
I20260709 12:26:55.387084 31058 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:55.388948 31058 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.389022 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:55.390331 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:55.394938  6228 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.397616 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:55.398362 30985 raft_consensus.cc:2393] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 22: Already voted for candidate e9cccf7eb2a545ef8c2169b67ff261bf in this term.
I20260709 12:26:55.427356  6209 raft_consensus.cc:993] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:55.427990  6262 raft_consensus.cc:1081] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:55.430915 31141 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45364
I20260709 12:26:55.431413 31141 raft_consensus.cc:493] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:55.431710 31141 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.440663 31058 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.448328 31141 raft_consensus.cc:515] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:55.464892 30990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:55.465512 30990 raft_consensus.cc:3055] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:55.465829 30990 raft_consensus.cc:740] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:55.466610 30990 consensus_queue.cc:260] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:55.467700 30990 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.468909 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:55.469605 30985 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.470642 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f93e9d74965c45a19eb84bff3d1f9825" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:55.471277 31064 raft_consensus.cc:3060] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.475435 30886 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.479156 30990 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:55.480901 31695 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1c1f8d5241540fdb4fb4daa1691260f" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:55.487524 31695 raft_consensus.cc:3055] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:55.487857 31695 raft_consensus.cc:740] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:55.488142 31109 leader_election.cc:304] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:55.489035  5916 raft_consensus.cc:2804] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:55.489634  5916 raft_consensus.cc:697] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:55.488735 31695 consensus_queue.cc:260] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.490597  5916 consensus_queue.cc:237] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:55.486829 30880 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 323) took 18171 ms (client timeout 15000 ms). Trace:
I20260709 12:26:55.492730 31695 raft_consensus.cc:3060] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:55.495641 30880 rpcz_store.cc:269] 0709 12:26:37.315015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.315250 (+   235us) service_pool.cc:224] Handling call
0709 12:26:55.486564 (+18171314us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:54.124587 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:54.125190 (+   603us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:54.125229 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:26:54.125249 (+    20us) tablet.cc:662] Decoding operations
0709 12:26:54.171001 (+ 45752us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:54.171099 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:54.171135 (+    36us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:26:54.173559 (+  2424us) tablet.cc:701] Row locks acquired
0709 12:26:54.173582 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:26:54.173745 (+   163us) write_op.cc:270] Start()
0709 12:26:54.173868 (+   123us) write_op.cc:276] Timestamp: P: 1783600014173693 usec, L: 0
0709 12:26:54.173895 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:26:54.177979 (+  4084us) log.cc:844] Serialized 11302 byte log entry
0709 12:26:54.296820 (+118841us) op_driver.cc:464] REPLICATION: finished
0709 12:26:54.594107 (+297287us) write_op.cc:301] APPLY: starting
0709 12:26:54.628249 (+ 34142us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:55.039995 (+411746us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:55.040136 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:55.228020 (+187884us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:55.248873 (+ 20853us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=58,mrs_lookups=0
0709 12:26:55.248952 (+    79us) write_op.cc:312] APPLY: finished
0709 12:26:55.252995 (+  4043us) log.cc:844] Serialized 252 byte log entry
0709 12:26:55.257952 (+  4957us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:55.259510 (+  1558us) write_op.cc:454] Released schema lock
0709 12:26:55.259598 (+    88us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:55.335337 (+ 75739us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:55.335371 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":169316,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":296947,"apply.run_cpu_time_us":40204,"apply.run_wall_time_us":741207,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125405,"num_ops":29,"prepare.queue_time_us":42093,"prepare.run_cpu_time_us":8136,"prepare.run_wall_time_us":54386,"raft.queue_time_us":26632,"raft.run_cpu_time_us":25408,"raft.run_wall_time_us":114728,"replication_time_us":122638,"thread_start_us":159556,"threads_started":4,"wal-append.queue_time_us":916}]]}
I20260709 12:26:55.507424 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:55.481686 31141 leader_election.cc:290] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:55.493049 31058 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:55.514866 31695 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:55.519191  6228 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:55.519621 31033 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:26:55.520632  5999 raft_consensus.cc:2804] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:55.521447  5999 raft_consensus.cc:697] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:55.522755 30985 raft_consensus.cc:2468] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:26:55.522531  5999 consensus_queue.cc:237] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.518245 31064 raft_consensus.cc:2468] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 20.
I20260709 12:26:55.562722 31143 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:26:55.592126 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:26:55.610637  6228 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:55.618594  6228 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:55.637216  6228 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:26:55.725430 30886 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 315) took 73335 ms (client timeout 15000 ms). Trace:
I20260709 12:26:55.726491 30888 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 23 to 26, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 26 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.730185 30888 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 20 to 26. New cstate: current_term: 26 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.731555 30888 catalog_manager.cc:5697] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 25, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 25 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.732798 30888 catalog_manager.cc:5697] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.737105 30888 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
W20260709 12:26:55.729226 30886 rpcz_store.cc:269] 0709 12:25:42.390129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.390461 (+   332us) service_pool.cc:224] Handling call
0709 12:26:55.725317 (+73334856us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:55.559857 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:55.579865 (+ 20008us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:55.579925 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:26:55.579946 (+    21us) tablet.cc:662] Decoding operations
0709 12:26:55.580357 (+   411us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:55.580462 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:55.580494 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:26:55.590507 (+ 10013us) tablet.cc:701] Row locks acquired
0709 12:26:55.590570 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:26:55.590785 (+   215us) write_op.cc:270] Start()
0709 12:26:55.594586 (+  3801us) write_op.cc:276] Timestamp: P: 1783600015590698 usec, L: 0
0709 12:26:55.594800 (+   214us) op_driver.cc:348] REPLICATION: starting
0709 12:26:55.595442 (+   642us) log.cc:844] Serialized 538 byte log entry
0709 12:26:55.653432 (+ 57990us) op_driver.cc:464] REPLICATION: finished
0709 12:26:55.660668 (+  7236us) write_op.cc:301] APPLY: starting
0709 12:26:55.660769 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:55.661654 (+   885us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:55.661763 (+   109us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:55.662515 (+   752us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:55.662678 (+   163us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:26:55.662728 (+    50us) write_op.cc:312] APPLY: finished
0709 12:26:55.662933 (+   205us) log.cc:844] Serialized 25 byte log entry
0709 12:26:55.663100 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:55.663181 (+    81us) write_op.cc:454] Released schema lock
0709 12:26:55.663228 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:55.663598 (+   370us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:55.663636 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":43059,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":644,"apply.run_cpu_time_us":2911,"apply.run_wall_time_us":2911,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":68629,"prepare.run_cpu_time_us":3444,"prepare.run_wall_time_us":35927,"replication_time_us":58450,"spinlock_wait_cycles":582528,"thread_start_us":75129,"threads_started":1}]]}
I20260709 12:26:55.740962 30888 catalog_manager.cc:5697] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 21, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 21 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:55.742512 30888 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.743639 30888 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.744819 30888 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:55.746409 30888 catalog_manager.cc:5697] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 23 to 24, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 24 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.747695 30888 catalog_manager.cc:5697] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 20 to 22, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 22 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.750130 30888 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 22 to 24, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 24 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.755172 30888 catalog_manager.cc:5697] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.756348 30888 catalog_manager.cc:5697] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 22, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 22 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.757802 30888 catalog_manager.cc:5697] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 20, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 20 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.759116 30888 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 13 to 24, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 24 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.760367 30888 catalog_manager.cc:5697] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.761624 30888 catalog_manager.cc:5697] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 19, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 19 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:55.763252 30888 catalog_manager.cc:5697] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 10 to 23, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 23 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:55.764704 30888 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.766067 30888 catalog_manager.cc:5697] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.767323 30888 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.768550 30888 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 13 to 22, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 22 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.773362 30888 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 18, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 18 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.778309 30888 catalog_manager.cc:5697] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.780352 30888 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.781495 30888 catalog_manager.cc:5697] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 21 to 22, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 22 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.784066 30888 catalog_manager.cc:5697] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 25, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 25 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.785339 30888 catalog_manager.cc:5697] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 20, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 20 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.786707 30888 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 22 to 25, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 25 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.787902 30888 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 22, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 22 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.789143 30888 catalog_manager.cc:5697] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.792809 30888 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.793983 30888 catalog_manager.cc:5697] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 23, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 23 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.796975 30888 catalog_manager.cc:5697] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 16, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 16 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.798269 30888 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 24, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 24 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.808586 30888 catalog_manager.cc:5697] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 21 to 24, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 24 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:55.810325 30888 catalog_manager.cc:5697] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 21, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 21 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:55.811669 30888 catalog_manager.cc:5697] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:55.812917 30888 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:55.851169  6058 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.851754  6058 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.857475 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:55.858139 31141 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 22.
I20260709 12:26:55.855386  6058 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:55.859157 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:26:55.864272 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:26:55.865671  6058 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:55.866074  6058 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.866443  6058 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.874585  6058 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:55.876503  6058 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Requested vote from peers ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:55.878607 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:55.879175 31143 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.879196 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:55.891754 31143 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9cccf7eb2a545ef8c2169b67ff261bf in term 23.
I20260709 12:26:55.902490 30958 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:55.906877  5951 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:55.907719  5951 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 LEADER]: Becoming Leader. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:55.908735  5951 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:56.029955 31064 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:56.032982  5916 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:56.048019 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 89392.1us
I20260709 12:26:56.069995 31697 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:26:56.072021 30985 raft_consensus.cc:1275] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:26:56.091187  6308 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:26:56.133713 31143 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:56.136376  6290 consensus_queue.cc:1059] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:56.137171  6229 raft_consensus.cc:3055] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:56.137553  6229 raft_consensus.cc:740] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:56.138355  6229 consensus_queue.cc:260] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:56.139472  6229 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:56.162446  6284 consensus_queue.cc:1048] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.004s
I20260709 12:26:56.166851 30990 raft_consensus.cc:1240] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 LEADER]: Rejecting Update request from peer a517aadb0ae04b89a13779d7898dfa0c for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:56.169263 30985 raft_consensus.cc:1275] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:56.172128  6317 consensus_queue.cc:1048] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:56.243357 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 48871.9us
I20260709 12:26:56.254523 30881 catalog_manager.cc:5697] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 12 to 18, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:56.257709 30881 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 19 to 24, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 24 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:26:56.263674 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 99036.9us
I20260709 12:26:56.278971 30881 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 13 to 17, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 17 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:56.283023 30881 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 14 to 18, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 18 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:56.285804 30881 catalog_manager.cc:5697] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 10 to 20, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 20 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:56.320497 30881 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 16 to 20, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 20 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:56.322422 30881 catalog_manager.cc:5697] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 12 to 15, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 15 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:56.324627 30881 catalog_manager.cc:5697] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 12 to 18, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:56.348924 30881 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 18 to 20, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 20 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:56.350713 30881 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 12 to 17, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 17 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:26:56.352921 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 88026.2us
I20260709 12:26:56.353259 30881 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 20 to 22, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 22 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:56.363588 30888 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48512 (request call id 324) took 2969 ms. Trace:
I20260709 12:26:56.386656 30888 rpcz_store.cc:276] 0709 12:26:53.387622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.387789 (+   167us) service_pool.cc:224] Handling call
0709 12:26:56.356905 (+2969116us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:55.835002 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:55.835643 (+   641us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:55.835684 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:26:55.835699 (+    15us) tablet.cc:662] Decoding operations
0709 12:26:55.838399 (+  2700us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:55.838454 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:55.838486 (+    32us) tablet.cc:685] Acquiring locks for 40 operations
0709 12:26:55.841092 (+  2606us) tablet.cc:701] Row locks acquired
0709 12:26:55.841107 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:26:55.841242 (+   135us) write_op.cc:270] Start()
0709 12:26:55.841354 (+   112us) write_op.cc:276] Timestamp: P: 1783600015841198 usec, L: 0
0709 12:26:55.841372 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:26:55.842117 (+   745us) log.cc:844] Serialized 15503 byte log entry
0709 12:26:55.859566 (+ 17449us) op_driver.cc:464] REPLICATION: finished
0709 12:26:55.862373 (+  2807us) write_op.cc:301] APPLY: starting
0709 12:26:55.862466 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:55.925218 (+ 62752us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:55.925322 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:56.177236 (+251914us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:56.177507 (+   271us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=80,key_file_lookups=80,delta_file_lookups=80,mrs_lookups=0
0709 12:26:56.177541 (+    34us) write_op.cc:312] APPLY: finished
0709 12:26:56.178149 (+   608us) log.cc:844] Serialized 340 byte log entry
0709 12:26:56.178899 (+   750us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:56.179209 (+   310us) write_op.cc:454] Released schema lock
0709 12:26:56.179315 (+   106us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:56.203347 (+ 24032us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:56.203381 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46774,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":2510,"apply.run_cpu_time_us":46444,"apply.run_wall_time_us":330987,"cfile_cache_hit":160,"cfile_cache_hit_bytes":167280,"num_ops":40,"prepare.queue_time_us":242,"prepare.run_cpu_time_us":7435,"prepare.run_wall_time_us":15854,"replication_time_us":18029}]]}
I20260709 12:26:56.436348 31697 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:26:56.436770 31697 pending_rounds.cc:85] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Aborting all ops after (but not including) 13
I20260709 12:26:56.436995 31697 pending_rounds.cc:107] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:26:56.437243 31697 raft_consensus.cc:2889] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:56.439379  6312 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:56.468837 31135 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Refusing update from remote peer e9cccf7eb2a545ef8c2169b67ff261bf: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:26:56.472436  6312 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:56.530031  6322 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:56.530664  6322 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:56.532927  6322 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:56.533552 31064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:26:56.534346 31064 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 24: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:56.542560 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:26:56.543197 31135 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:26:56.544533 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:56.545303  6322 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:56.545689  6322 raft_consensus.cc:493] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:26:56.546053  6322 raft_consensus.cc:3060] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:56.561466  6322 raft_consensus.cc:515] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:56.564002  6322 leader_election.cc:290] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:56.564627 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:56.588500 31058 raft_consensus.cc:2393] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a517aadb0ae04b89a13779d7898dfa0c in current term 24: Already voted for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in this term.
I20260709 12:26:56.564939 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "330f8038ee4e498ebefd9d773f4e7199" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:56.610636 31135 raft_consensus.cc:2468] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:26:56.611888 31664 leader_election.cc:304] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:56.623632  6290 raft_consensus.cc:2804] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:56.624172  6290 raft_consensus.cc:697] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:26:56.633162  6290 consensus_queue.cc:237] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:56.643183 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:56.651872 30886 catalog_manager.cc:5697] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 19, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 19 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:56.655295 30886 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 22 to 24, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 24 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:56.674553 30886 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 27, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 27 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:56.676656 30886 catalog_manager.cc:5697] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 20, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 20 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:56.678037 30886 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 20 to 21, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 21 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:56.734052 30886 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 16 to 17, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 17 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } }
I20260709 12:26:56.879446 30881 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 319) took 6801 ms. Trace:
I20260709 12:26:56.880728 30881 rpcz_store.cc:276] 0709 12:26:50.090849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.091869 (+  1020us) service_pool.cc:224] Handling call
0709 12:26:50.177788 (+ 85919us) spinlock_profiling.cc:243] Waited 57.4 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fe090 00007f2eeb89cf3a 00007f2ef1652e9c 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:56.879348 (+6701560us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:56.527317 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:56.529626 (+  2309us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:56.529668 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:26:56.529686 (+    18us) tablet.cc:662] Decoding operations
0709 12:26:56.530713 (+  1027us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:56.530791 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:56.530818 (+    27us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:26:56.531759 (+   941us) tablet.cc:701] Row locks acquired
0709 12:26:56.531779 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:26:56.531907 (+   128us) write_op.cc:270] Start()
0709 12:26:56.532033 (+   126us) write_op.cc:276] Timestamp: P: 1783600016531866 usec, L: 0
0709 12:26:56.557521 (+ 25488us) op_driver.cc:348] REPLICATION: starting
0709 12:26:56.558182 (+   661us) log.cc:844] Serialized 4339 byte log entry
0709 12:26:56.605395 (+ 47213us) op_driver.cc:464] REPLICATION: finished
0709 12:26:56.612607 (+  7212us) write_op.cc:301] APPLY: starting
0709 12:26:56.612687 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:56.618172 (+  5485us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:56.618282 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:56.638000 (+ 19718us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:56.638238 (+   238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 12:26:56.638282 (+    44us) write_op.cc:312] APPLY: finished
0709 12:26:56.638571 (+   289us) log.cc:844] Serialized 105 byte log entry
0709 12:26:56.638724 (+   153us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:56.638870 (+   146us) write_op.cc:454] Released schema lock
0709 12:26:56.638918 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:56.664374 (+ 25456us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:56.664414 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":43909,"reported_tablets":48,"spinlock_wait_cycles":132118272,"child_traces":[["op",{"apply.queue_time_us":6968,"apply.run_cpu_time_us":13316,"apply.run_wall_time_us":34509,"cfile_cache_hit":44,"cfile_cache_hit_bytes":46002,"num_ops":11,"prepare.queue_time_us":153507,"prepare.run_cpu_time_us":4119,"prepare.run_wall_time_us":38809,"replication_time_us":47732,"thread_start_us":152733,"threads_started":1}]]}
I20260709 12:26:56.916388 30882 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 15 to 16, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 16 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:56.928442 30882 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 10 to 15, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 15 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:56.991190 30880 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 20 to 21, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 21 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:56.993844 30880 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:57.029240 30992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:57.029919 30992 raft_consensus.cc:606] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:57.031373 30880 catalog_manager.cc:5697] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 17 to 18, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:57.034660 30880 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 15 to 17, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 17 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:26:57.044983 30882 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 318) took 28476 ms (client timeout 15000 ms). Trace:
I20260709 12:26:57.049238 30992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:57.052088 30992 raft_consensus.cc:606] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:57.055744 31695 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba8a01bf2a514da2805eb75a7c28c412"
dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:59170
I20260709 12:26:57.056350 31695 raft_consensus.cc:606] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:57.056843 31695 raft_consensus.cc:614] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:57.070868 30992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:57.071465 30992 raft_consensus.cc:606] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:57.073545  5951 raft_consensus.cc:993] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:57.074018  5951 raft_consensus.cc:1081] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:57.074465 30992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:57.075152 30992 raft_consensus.cc:606] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:57.075568 31062 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:26:57.076148 31062 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:57.076493 31062 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:57.085440 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:57.087903 31062 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:57.053251 30882 rpcz_store.cc:269] 0709 12:26:28.569073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.569262 (+   189us) service_pool.cc:224] Handling call
0709 12:26:29.110659 (+541397us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b5400011228. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2ef1a86331 00007f2ef1a813fc 00007f2ef1652e84 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:57.044902 (+27934243us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:56.938106 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:56.938651 (+   545us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:56.938681 (+    30us) write_op.cc:435] Acquired schema lock
0709 12:26:56.938694 (+    13us) tablet.cc:662] Decoding operations
0709 12:26:56.939113 (+   419us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:56.939161 (+    48us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:56.939190 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:26:56.939514 (+   324us) tablet.cc:701] Row locks acquired
0709 12:26:56.939526 (+    12us) write_op.cc:260] PREPARE: finished
0709 12:26:56.939665 (+   139us) write_op.cc:270] Start()
0709 12:26:56.939784 (+   119us) write_op.cc:276] Timestamp: P: 1783600016939619 usec, L: 0
0709 12:26:56.939799 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:26:56.940349 (+   550us) log.cc:844] Serialized 913 byte log entry
0709 12:26:56.942508 (+  2159us) op_driver.cc:464] REPLICATION: finished
0709 12:26:56.942933 (+   425us) write_op.cc:301] APPLY: starting
0709 12:26:56.943017 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:56.944241 (+  1224us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:56.944333 (+    92us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:56.945492 (+  1159us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:56.945654 (+   162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 12:26:56.945702 (+    48us) write_op.cc:312] APPLY: finished
0709 12:26:56.945887 (+   185us) log.cc:844] Serialized 33 byte log entry
0709 12:26:56.946064 (+   177us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:56.946150 (+    86us) write_op.cc:454] Released schema lock
0709 12:26:56.946192 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:56.946565 (+   373us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:56.946594 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":48,"spinlock_wait_cycles":33024384,"child_traces":[["op",{"apply.queue_time_us":221,"apply.run_cpu_time_us":3608,"apply.run_wall_time_us":3607,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":2510,"prepare.run_wall_time_us":2510,"replication_time_us":2602}]]}
I20260709 12:26:57.114807 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:57.117383 30992 raft_consensus.cc:3055] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:57.117787 30992 raft_consensus.cc:740] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:57.118575 30992 consensus_queue.cc:260] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:57.119683 30992 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:57.116134 30881 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 21 to 23, leader changed from ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) to e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65). New cstate: current_term: 23 leader_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:57.122948 31058 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:26:57.124174 30881 catalog_manager.cc:5697] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 21 to 22, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 22 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:26:57.125362  6229 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:57.136248 30880 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 204) took 3433 ms. Trace:
I20260709 12:26:57.125667 30881 catalog_manager.cc:5697] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 17 to 24, leader changed from a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 24 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:57.139312 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:57.140369 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:57.165637 31135 raft_consensus.cc:1275] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:26:57.170900  6228 consensus_queue.cc:1048] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:57.136682 30880 rpcz_store.cc:276] 0709 12:26:53.702626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.505192 (+1802566us) service_pool.cc:224] Handling call
0709 12:26:57.136163 (+1630971us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:57.045571 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:57.046111 (+   540us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:57.046163 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:26:57.046181 (+    18us) tablet.cc:662] Decoding operations
0709 12:26:57.046822 (+   641us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:57.046877 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:57.046909 (+    32us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:26:57.047317 (+   408us) tablet.cc:701] Row locks acquired
0709 12:26:57.047333 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:26:57.047476 (+   143us) write_op.cc:270] Start()
0709 12:26:57.047583 (+   107us) write_op.cc:276] Timestamp: P: 1783600017047432 usec, L: 0
0709 12:26:57.047603 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:26:57.055982 (+  8379us) log.cc:844] Serialized 1674 byte log entry
0709 12:26:57.057867 (+  1885us) op_driver.cc:464] REPLICATION: finished
0709 12:26:57.058414 (+   547us) spinlock_profiling.cc:243] Waited 7.71 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fd571 00007f2eeb88fe28 00007f2eef2ca581 00007f2eef2c9d58 00007f2eef2ce9f1 00007f2eef2ce9a9 00007f2eef2ce939 00007f2eef2ce901 00007f2eef2cdb2d 0000000000388634 0000000000388479
0709 12:26:57.058674 (+   260us) write_op.cc:301] APPLY: starting
0709 12:26:57.058749 (+    75us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:57.060848 (+  2099us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:57.060872 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:57.063328 (+  2456us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:57.063500 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:26:57.063544 (+    44us) write_op.cc:312] APPLY: finished
0709 12:26:57.063748 (+   204us) log.cc:844] Serialized 49 byte log entry
0709 12:26:57.064445 (+   697us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:57.064530 (+    85us) write_op.cc:454] Released schema lock
0709 12:26:57.064568 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:57.064904 (+   336us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:57.064930 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29099,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":528,"apply.run_cpu_time_us":5550,"apply.run_wall_time_us":6180,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16728,"num_ops":4,"prepare.queue_time_us":5896,"prepare.run_cpu_time_us":5212,"prepare.run_wall_time_us":13003,"replication_time_us":10131,"spinlock_wait_cycles":17875712}]]}
I20260709 12:26:57.186060 30992 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 28.
I20260709 12:26:57.198124 31033 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:57.203248  6284 raft_consensus.cc:2804] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:57.203958  6284 raft_consensus.cc:697] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 28 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:57.205031  6284 consensus_queue.cc:237] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:57.217029 31062 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 28 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:57.275449 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 68777.6us
W20260709 12:26:57.520859  6333 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:26:57.523093  6332 log.cc:927] Time spent T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:26:57.543251  5951 raft_consensus.cc:993] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:57.543787  5951 raft_consensus.cc:1081] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:57.541721 30881 catalog_manager.cc:5697] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:26:57.563760 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 93567us
W20260709 12:26:57.579348 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 195696us
I20260709 12:26:57.625697  6348 raft_consensus.cc:993] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
W20260709 12:26:57.654831 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 282632us
I20260709 12:26:57.655871  5951 raft_consensus.cc:1081] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:57.661056 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:26:57.661609 31060 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:57.661944 31060 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:57.676272 31058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:26:57.676854 31058 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:57.677222 31058 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:57.684147 31058 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:57.685024 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 308310us
I20260709 12:26:57.686894 31058 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:57.699431 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:57.703658 30985 raft_consensus.cc:3055] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:57.703984 30985 raft_consensus.cc:740] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:57.716099 30985 consensus_queue.cc:260] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:57.769505 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 75093.2us
W20260709 12:26:57.782874 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 90640.7us
W20260709 12:26:57.812120 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 125754us
W20260709 12:26:57.814991 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 62114.7us
W20260709 12:26:57.867105 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49938.6us
I20260709 12:26:57.924207 30985 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:57.936046 30882 catalog_manager.cc:5697] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 27 to 28, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 28 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:57.950579 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 133848us
I20260709 12:26:57.949012 31060 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:57.961699 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 64770.3us
I20260709 12:26:57.968021 31060 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:26:57.975816 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 187067us
I20260709 12:26:57.983635 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4162f35b54e4acc8abf0a1f9fc084a9" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:57.984257 31143 raft_consensus.cc:3060] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:58.031044 30958 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 61702.9us
W20260709 12:26:58.034984 30955 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 58307.2us
I20260709 12:26:58.035588  6349 raft_consensus.cc:993] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:58.036082  6349 raft_consensus.cc:1081] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:58.038928 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:58.039547 30984 raft_consensus.cc:3055] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:58.039826 30984 raft_consensus.cc:740] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:58.040598 30984 consensus_queue.cc:260] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.041750 30984 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:58.041998 31064 raft_consensus.cc:1240] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:58.046095 30990 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 28 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:58.047976 31060 raft_consensus.cc:1240] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:58.059834 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:26:58.060390 31060 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:58.060775 31060 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:26:58.061642 30885 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 317) took 32102 ms (client timeout 15000 ms). Trace:
I20260709 12:26:58.063781 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f311574d640f4ff69dd8e9525a8fe8c1" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:58.064369 31140 raft_consensus.cc:3060] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.069096 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aff3791e3404a7e92be14f04f9674ec" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:58.069664 31135 raft_consensus.cc:3060] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:58.073539  6336 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:58.091894 30984 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 25.
I20260709 12:26:58.013232 30985 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
W20260709 12:26:58.062034 30885 rpcz_store.cc:269] 0709 12:26:25.959349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.969287 (+  9938us) service_pool.cc:224] Handling call
0709 12:26:58.061462 (+32092175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120500,"reported_tablets":47}
I20260709 12:26:58.103103 31140 raft_consensus.cc:2468] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:58.106359 31135 raft_consensus.cc:2468] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 25.
I20260709 12:26:58.126724 31060 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.149003 31068 raft_consensus.cc:1240] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Rejecting Update request from peer e9cccf7eb2a545ef8c2169b67ff261bf for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:58.153219 31060 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 27 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:58.158396 31033 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:58.159314  6336 raft_consensus.cc:2804] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:58.159781  6336 raft_consensus.cc:697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:58.160634  6336 consensus_queue.cc:237] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.162859 30985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:58.163410 30985 raft_consensus.cc:3055] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:58.163676 30985 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:58.164381 30985 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.165776 30985 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:58.168468 31033 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:58.169301  5999 raft_consensus.cc:2804] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:58.169831  5999 raft_consensus.cc:697] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:58.171252 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:58.171794 31135 raft_consensus.cc:3060] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:58.142948 31143 raft_consensus.cc:2468] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 28.
I20260709 12:26:58.175133 31141 raft_consensus.cc:1275] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 28 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:58.177044 30985 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 27.
I20260709 12:26:58.177100  5999 consensus_queue.cc:237] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.182850 31033 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:58.183647  6284 raft_consensus.cc:2804] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:58.183101 31135 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 27.
I20260709 12:26:58.184336  6284 raft_consensus.cc:697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:58.185240  6284 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.187696  6335 consensus_queue.cc:1048] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
W20260709 12:26:58.438930 30887 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 317) took 52787 ms (client timeout 15000 ms). Trace:
W20260709 12:26:58.440447 30887 rpcz_store.cc:269] 0709 12:26:05.650926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.651135 (+   209us) service_pool.cc:224] Handling call
0709 12:26:58.438713 (+52787578us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":48}
I20260709 12:26:58.464423 30881 catalog_manager.cc:5697] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 24 to 25, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 25 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:58.466197 30881 catalog_manager.cc:5697] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 18, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 18 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:58.467836 30881 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 26 to 27, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 27 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:58.566260 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:58.566004 30889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48516 (request call id 315) took 93549 ms (client timeout 15000 ms). Trace:
W20260709 12:26:58.578696 30889 rpcz_store.cc:269] 0709 12:25:25.016618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.016851 (+   233us) service_pool.cc:224] Handling call
0709 12:26:58.565775 (+93548924us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":250159,"reported_tablets":48,"spinlock_wait_cycles":1792}
I20260709 12:26:58.606431 30985 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:58.607820  6284 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:58.623401 30984 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:26:58.626754  6284 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:58.674402 31141 raft_consensus.cc:1275] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:58.680295  6334 consensus_queue.cc:1048] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:58.698379 31143 raft_consensus.cc:1275] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:26:58.699904  6334 consensus_queue.cc:1048] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:58.709569 30984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:58.710111 30984 raft_consensus.cc:606] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:58.730194 30984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:58.730774 30984 raft_consensus.cc:606] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:58.731412 31135 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:26:58.732954  6334 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:58.745332 30992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:58.745877 30992 raft_consensus.cc:606] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Received request to transfer leadership to TS ce27a72886964e4287059a1d66bd5b6b
I20260709 12:26:58.754978 30992 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:26:58.756999  6334 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:58.760684 30985 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:58.761209 30985 raft_consensus.cc:606] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:58.773674 30984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c"
dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
mode: GRACEFUL
new_leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:55468
I20260709 12:26:58.774209 30984 raft_consensus.cc:606] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Received request to transfer leadership to TS 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:26:58.775367 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:58.778088 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:58.779107 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:58.813514  6370 raft_consensus.cc:993] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:58.814041  6370 raft_consensus.cc:1081] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:58.816017 31067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:26:58.816557 31067 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:58.816941 31067 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.836300  6058 raft_consensus.cc:993] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:58.836850  6058 raft_consensus.cc:1081] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:58.845068 31067 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.871547 31067 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:58.866448 31143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45364
I20260709 12:26:58.872483 31143 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:58.872872 31143 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:58.875366 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:58.875945 31141 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.878895 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39b66c047b4e42afad80738552eee0dc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:58.879511 30992 raft_consensus.cc:3055] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:58.879796 30992 raft_consensus.cc:740] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:58.880538 30992 consensus_queue.cc:260] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.881796 30992 raft_consensus.cc:3060] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.897823 31141 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:58.899791 31143 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:58.901381 31033 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:26:58.902100  6335 raft_consensus.cc:2804] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:58.902412 31143 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:26:58.902638  6335 raft_consensus.cc:697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:58.903530  6335 consensus_queue.cc:237] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.906239 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:26:58.906904 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72d9d9cc8be6411883e9f3f17ef865eb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:58.907523 30984 raft_consensus.cc:3055] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:58.907828 30984 raft_consensus.cc:740] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:58.906833 31697 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:58.908591 30984 consensus_queue.cc:260] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:58.909883 30984 raft_consensus.cc:3060] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:58.922477  6362 raft_consensus.cc:993] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:58.922930 30984 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:58.923028  6058 raft_consensus.cc:1081] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Signalling peer ce27a72886964e4287059a1d66bd5b6b to start an election
I20260709 12:26:58.924156 31109 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:58.924563 30992 raft_consensus.cc:2468] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:58.925963  6368 raft_consensus.cc:2804] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:58.926570  6368 raft_consensus.cc:697] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:58.927480  6368 consensus_queue.cc:237] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:26:58.932653 31697 raft_consensus.cc:2468] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:26:58.942170  6058 raft_consensus.cc:993] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:58.946460 31143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758"
dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
 from {username='slave'} at 127.0.0.1:45364
I20260709 12:26:58.947633 31143 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:58.946053 30880 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 205) took 1590 ms. Trace:
I20260709 12:26:58.949173  6058 raft_consensus.cc:1081] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:58.950253 30880 rpcz_store.cc:276] 0709 12:26:57.355965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.356144 (+   179us) service_pool.cc:224] Handling call
0709 12:26:58.945989 (+1589845us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19175,"reported_tablets":10}
I20260709 12:26:58.951542 31143 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:58.951607 31067 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:26:58.952527 31067 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:58.952859 31067 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.959288 31067 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.961952 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:58.964650 30984 raft_consensus.cc:3055] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:58.964965 30984 raft_consensus.cc:740] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:58.965708 30984 consensus_queue.cc:260] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:58.966812 30984 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.967693 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dbea78f5c04486a0fc3f17e2260e2c" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:58.968228 31141 raft_consensus.cc:3060] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.975517 30984 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:58.961073 31067 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:58.979430 31033 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:58.991544  6284 raft_consensus.cc:2804] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:58.992487  6284 raft_consensus.cc:697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:58.993396  6284 consensus_queue.cc:237] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:59.001703 30883 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48508 (request call id 318) took 11400 ms (client timeout 15000 ms). Trace:
I20260709 12:26:59.005023 31143 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:26:59.007583 30884 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59312 (request call id 203) took 23057 ms (client timeout 15000 ms). Trace:
W20260709 12:26:59.022377 30884 rpcz_store.cc:269] 0709 12:26:35.950615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.950819 (+   204us) service_pool.cc:224] Handling call
0709 12:26:37.075849 (+1125030us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fe090 00007f2eeb89cf3a 00007f2ef1652e9c 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:59.007394 (+21931545us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55401,"reported_tablets":36,"spinlock_wait_cycles":2249383296}
I20260709 12:26:59.022854 30880 catalog_manager.cc:5697] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 21 to 22, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 22 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:59.002161 30883 rpcz_store.cc:269] 0709 12:26:47.601600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.601926 (+   326us) service_pool.cc:224] Handling call
0709 12:26:48.344992 (+743066us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b54003101d4. stack: 00007f2ee8775343 00007f2ee8775240 00007f2ee81041d4 00007f2ef1e42304 00007f2ef1e42299 00007f2eeb7fe090 00007f2eeb89cf3a 00007f2ef1652e9c 00007f2ef175104e 00007f2eefc122a4 00007f2eefc12232 00007f2eefc12161 00007f2eefc120dc 00007f2eefc11392 00007f2ee9bcf48c 00007f2ee9bce8c6
0709 12:26:59.001506 (+10656514us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51999,"reported_tablets":47,"spinlock_wait_cycles":3173632}
I20260709 12:26:59.024868 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:26:59.025446 31067 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.026453 31143 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:26:59.031962 30992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676d422dffb24e3cb16409a80bb28758" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:59.032532 30992 raft_consensus.cc:3055] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:59.032835 30992 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:59.033588 30992 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:59.034798 30992 raft_consensus.cc:3060] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.041667 30992 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:59.042591 31109 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:59.043370  6353 raft_consensus.cc:2804] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:59.043833  6353 raft_consensus.cc:697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:26:59.044661  6353 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:59.062525 31141 raft_consensus.cc:2468] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:26:59.068353  6264 raft_consensus.cc:993] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: : Instructing follower 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:59.068812  6264 raft_consensus.cc:1081] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Signalling peer 2c4511d66b474ee0aa41c9b1f03b66ab to start an election
I20260709 12:26:59.041901 31067 raft_consensus.cc:2468] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:26:59.070571 31060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc"
dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
 from {username='slave'} at 127.0.0.1:51946
I20260709 12:26:59.071092 31060 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:59.071441 31060 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.084627 31060 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:59.092123 30883 catalog_manager.cc:5697] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 18 to 19, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 19 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:59.093005 30984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf"
I20260709 12:26:59.093676 30984 raft_consensus.cc:3055] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:59.093961 30984 raft_consensus.cc:740] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e9cccf7eb2a545ef8c2169b67ff261bf, State: Running, Role: LEADER
I20260709 12:26:59.095459 31060 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:26:59.094702 30984 consensus_queue.cc:260] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:59.097194 31141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "924fd2989ab145f09d211890a85a03fc" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:26:59.115063 31141 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.115306 30984 raft_consensus.cc:3060] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.125283 30984 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:59.126253 31033 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf; no voters: 
I20260709 12:26:59.127170  6284 raft_consensus.cc:2804] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:59.127892  6284 raft_consensus.cc:697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:26:59.128791  6284 consensus_queue.cc:237] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:26:59.136487 31141 raft_consensus.cc:2468] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 22.
I20260709 12:26:59.320132 30883 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 22 to 23, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:59.322939 31697 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:59.324996 30880 catalog_manager.cc:5697] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 17 to 18, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 18 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:59.327840 30880 catalog_manager.cc:5697] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 21 to 22, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 22 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:59.326613  6347 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 12:26:59.364830 31141 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:59.369230  6334 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:59.534307 30985 raft_consensus.cc:1275] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:26:59.519634  6399 log.cc:927] Time spent T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:59.548120  6335 consensus_queue.cc:1048] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:59.549839 31143 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:26:59.584347 30985 raft_consensus.cc:1275] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:59.590883 30985 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:59.617523  6407 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:26:59.622797  6368 consensus_queue.cc:1048] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:59.628566 30992 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:59.635111 31060 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:59.635032  6368 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:26:59.642169  6347 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:59.652012  6406 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
W20260709 12:26:59.741750 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 57237.9us
I20260709 12:26:59.763412 31141 raft_consensus.cc:1275] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:59.808413  6408 consensus_queue.cc:1048] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.120s
I20260709 12:26:59.996016 30983 raft_consensus.cc:1275] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:27:00.131335 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 313146us
W20260709 12:27:00.231272 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 47629.8us
W20260709 12:27:00.242340 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 56641.5us
I20260709 12:27:00.199270 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:00.280141 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 50265.2us
W20260709 12:27:00.402415 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 115099us
W20260709 12:27:00.403975 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 114784us
W20260709 12:27:00.401362 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 108642us
W20260709 12:27:00.407241 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 111219us
I20260709 12:27:00.300595  6284 consensus_queue.cc:1048] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.128s
W20260709 12:27:00.509994 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 82047.5us
W20260709 12:27:00.560472 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 153066us
W20260709 12:27:00.534473  6420 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: Append to log took a long time: real 0.076s	user 0.006s	sys 0.003s
W20260709 12:27:00.580595 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 83176.2us
W20260709 12:27:00.599195  6443 log.cc:927] Time spent T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 12:27:00.702425 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 116306us
I20260709 12:27:00.824457 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:00.826296 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:00.827076 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:00.892555 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:00.948819 29845 tablet_server.cc:179] TabletServer@127.29.37.65:0 shutting down...
W20260709 12:27:00.965070 31109 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:00.987831 31109 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:00.988678 31033 consensus_peers.cc:597] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.021318 31033 consensus_peers.cc:597] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.054584 31109 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.057195 31109 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.087054 31033 consensus_peers.cc:597] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.086700 31109 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.094421 31109 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.116417 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.119410 31109 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.119920 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.127440 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.144527 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.147874 31109 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.181707 31033 consensus_peers.cc:597] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.203210 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.207031 31109 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.208622 31033 consensus_peers.cc:597] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.216879 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.235728 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:01.245411 29845 tablet_replica.cc:333] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
W20260709 12:27:01.246109 31033 consensus_peers.cc:597] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.251127 29845 raft_consensus.cc:2243] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.258774 29845 raft_consensus.cc:2272] T a2c77b15766d44f585ab3cb57444a324 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:01.268111 31109 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.277082 31033 consensus_peers.cc:597] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.280678 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.286695 29845 tablet_replica.cc:333] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.287849 29845 raft_consensus.cc:2243] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.288899 29845 raft_consensus.cc:2272] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.291744 29845 tablet_replica.cc:333] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.297181 29845 raft_consensus.cc:2243] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.301818 29845 raft_consensus.cc:2272] T d521786b7ccd41da81f7fd0d175bf582 P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.304625 29845 tablet_replica.cc:333] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.305416 29845 raft_consensus.cc:2243] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.305977 29845 raft_consensus.cc:2272] T 76399531471a4790a56d52a6ea052864 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.308979 29845 tablet_replica.cc:333] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.309949 29845 raft_consensus.cc:2243] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.310539 29845 raft_consensus.cc:2272] T d50d1465e4b84136bf90de700a93ef2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.313052 29845 tablet_replica.cc:333] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
W20260709 12:27:01.313153 31109 consensus_peers.cc:597] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.313855 29845 raft_consensus.cc:2243] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.314558 29845 raft_consensus.cc:2272] T 95980265645c4f7a8d6b201f0ae58174 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.317106 29845 tablet_replica.cc:333] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.317915 29845 raft_consensus.cc:2243] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.318482 29845 raft_consensus.cc:2272] T 1aff3791e3404a7e92be14f04f9674ec P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.321161 29845 tablet_replica.cc:333] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.322101 29845 raft_consensus.cc:2243] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.323238 29845 raft_consensus.cc:2272] T 635264f51b2d4410a1da9b76606a6350 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.325965 29845 tablet_replica.cc:333] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.328336 29845 raft_consensus.cc:2243] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.329317 29845 raft_consensus.cc:2272] T 544dffe603ca4dce9d6697ea6b15c925 P e9cccf7eb2a545ef8c2169b67ff261bf [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.332056 29845 tablet_replica.cc:333] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.332849 29845 raft_consensus.cc:2243] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.333403 29845 raft_consensus.cc:2272] T f311574d640f4ff69dd8e9525a8fe8c1 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.335867 29845 tablet_replica.cc:333] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.336647 29845 raft_consensus.cc:2243] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.337657 29845 raft_consensus.cc:2272] T 4950edf9dfb9412981a0b325bf866deb P e9cccf7eb2a545ef8c2169b67ff261bf [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.340152 29845 tablet_replica.cc:333] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.340953 29845 raft_consensus.cc:2243] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.341562 29845 raft_consensus.cc:2272] T 044e61a321dd4521be7f9ee796ca44dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.343964 29845 tablet_replica.cc:333] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.344696 29845 raft_consensus.cc:2243] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.345388 29845 raft_consensus.cc:2272] T 4a3e0211fca8429086a4403f1ec42a59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.348074 29845 tablet_replica.cc:333] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.353367 29845 raft_consensus.cc:2243] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.354460 29845 raft_consensus.cc:2272] T d48e42d074cd496f996172cff1a0c835 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.357069 29845 tablet_replica.cc:333] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.357875 29845 raft_consensus.cc:2243] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.358928 29845 raft_consensus.cc:2272] T 0afdb545bb8f485993259007a00c1841 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.361419 29845 tablet_replica.cc:333] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.362143 29845 raft_consensus.cc:2243] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.362706 29845 raft_consensus.cc:2272] T 676d422dffb24e3cb16409a80bb28758 P e9cccf7eb2a545ef8c2169b67ff261bf [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.365047 29845 tablet_replica.cc:333] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
W20260709 12:27:01.369956 31033 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:01.374269 31033 consensus_peers.cc:597] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.376696 29845 raft_consensus.cc:2243] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.377828 29845 raft_consensus.cc:2272] T d8e9bb54e628458a94d623c051d36c1a P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.380574 29845 tablet_replica.cc:333] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.381587 29845 raft_consensus.cc:2243] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.382726 29845 raft_consensus.cc:2272] T 82399103216c4d5e8d2acdbbf3cee84d P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:01.390151 31664 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.393911 29845 tablet_replica.cc:333] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.394891 29845 raft_consensus.cc:2243] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.395538 29845 raft_consensus.cc:2272] T 9dab2cd673924dc2a2f8622e76701947 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.398065 29845 tablet_replica.cc:333] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
W20260709 12:27:01.398586 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.398964 29845 raft_consensus.cc:2243] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 LEADER]: Raft consensus shutting down.
W20260709 12:27:01.404011 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.405200 29845 raft_consensus.cc:2272] T 363b01e660fa4b36aa4a5112bab1ac09 P e9cccf7eb2a545ef8c2169b67ff261bf [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.408182 29845 tablet_replica.cc:333] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.409088 29845 raft_consensus.cc:2243] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.409711 29845 raft_consensus.cc:2272] T c6dbea78f5c04486a0fc3f17e2260e2c P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.413255 29845 tablet_replica.cc:333] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.414083 29845 raft_consensus.cc:2243] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.415174 29845 raft_consensus.cc:2272] T 83f49f3c3cc44994877e69c3ff83090e P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.422272 29845 tablet_replica.cc:333] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.423175 29845 raft_consensus.cc:2243] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:01.428364 31033 consensus_peers.cc:597] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.428871 29845 raft_consensus.cc:2272] T 563f5eec53c14273959c9883d2423cc9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.441624 29845 tablet_replica.cc:333] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.446921 29845 raft_consensus.cc:2243] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.449530 29845 raft_consensus.cc:2272] T 82382d1e93d244cd8307164a5ac891b0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.474514 29845 tablet_replica.cc:333] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.475386 29845 raft_consensus.cc:2243] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.476359 29845 raft_consensus.cc:2272] T 39b66c047b4e42afad80738552eee0dc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.479096 29845 tablet_replica.cc:333] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.479967 29845 raft_consensus.cc:2243] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.480638 29845 raft_consensus.cc:2272] T 5bd1ada4571045f6ba305e207cccddcf P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.493439 29845 tablet_replica.cc:333] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
W20260709 12:27:01.500742 31033 consensus_peers.cc:597] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.502039 29845 raft_consensus.cc:2243] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.502766 29845 raft_consensus.cc:2272] T f93e9d74965c45a19eb84bff3d1f9825 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.514518 29845 tablet_replica.cc:333] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.518949 29845 raft_consensus.cc:2243] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.527215 29845 raft_consensus.cc:2272] T 13959daae3c149d6ae8248d1f3ca6e59 P e9cccf7eb2a545ef8c2169b67ff261bf [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:01.533578 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:01.542743 29845 tablet_replica.cc:333] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.570964 29845 raft_consensus.cc:2243] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.571736 29845 raft_consensus.cc:2272] T e98467ef2b2b4bab954b704025d182b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.583513 29845 tablet_replica.cc:333] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.602815 29845 raft_consensus.cc:2243] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.608247 29845 raft_consensus.cc:2272] T a7f0812b392640ecab5f5257bb0e41ee P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.612092 29845 tablet_replica.cc:333] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.617616 29845 raft_consensus.cc:2243] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.623200 29845 raft_consensus.cc:2272] T 2439e7934b98446ca4e7d82381e8a499 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.631546 29845 tablet_replica.cc:333] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.638661 29845 raft_consensus.cc:2243] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.639379 29845 raft_consensus.cc:2272] T 76fe9dfd8b4c4785a354982ad7e1faa4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.642107 29845 tablet_replica.cc:333] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.658651 29845 raft_consensus.cc:2243] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.659825 29845 raft_consensus.cc:2272] T bc9ac1b8eaf94e99a3eee16d0ca65113 P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.674475 29845 tablet_replica.cc:333] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.675305 29845 raft_consensus.cc:2243] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.675925 29845 raft_consensus.cc:2272] T 60b6187f32774daba89763099dabeb2a P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.682199 29845 tablet_replica.cc:333] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.683107 29845 raft_consensus.cc:2243] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.683876 29845 raft_consensus.cc:2272] T 205fb3f3a3844073a49da13542da2989 P e9cccf7eb2a545ef8c2169b67ff261bf [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.693084 29845 tablet_replica.cc:333] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.710881 29845 raft_consensus.cc:2243] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.714836 29845 raft_consensus.cc:2272] T e3de96a6f20142c0a00ff9ee12bcc86c P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.717721 29845 tablet_replica.cc:333] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.719282 29845 raft_consensus.cc:2243] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.719951 29845 raft_consensus.cc:2272] T ba8a01bf2a514da2805eb75a7c28c412 P e9cccf7eb2a545ef8c2169b67ff261bf [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.735085 29845 tablet_replica.cc:333] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.735898 29845 raft_consensus.cc:2243] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.736532 29845 raft_consensus.cc:2272] T dd024d82eb9c473ba31252aeeb287fc0 P e9cccf7eb2a545ef8c2169b67ff261bf [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.739705 29845 tablet_replica.cc:333] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.740561 29845 raft_consensus.cc:2243] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.741235 29845 raft_consensus.cc:2272] T 8e81e4942d094afda44355ba1d9c7735 P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.744059 29845 tablet_replica.cc:333] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.744866 29845 raft_consensus.cc:2243] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.750614 29845 raft_consensus.cc:2272] T b8ba55d3e2574f7a890f7a956ba4e74f P e9cccf7eb2a545ef8c2169b67ff261bf [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.753401 29845 tablet_replica.cc:333] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.767719 29845 raft_consensus.cc:2243] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.768252 29845 raft_consensus.cc:2272] T b4162f35b54e4acc8abf0a1f9fc084a9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.778833 29845 tablet_replica.cc:333] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.783597 29845 raft_consensus.cc:2243] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:27:01.787086 29845 raft_consensus.cc:2272] T 7ea7681ba29145eba9db5bc137cf18b4 P e9cccf7eb2a545ef8c2169b67ff261bf [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.791219 29845 tablet_replica.cc:333] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.804049 29845 raft_consensus.cc:2243] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.804790 29845 raft_consensus.cc:2272] T b1c1f8d5241540fdb4fb4daa1691260f P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.812216 29845 tablet_replica.cc:333] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.814436 29845 raft_consensus.cc:2243] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.822664 29845 raft_consensus.cc:2272] T 924fd2989ab145f09d211890a85a03fc P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.835783 29845 tablet_replica.cc:333] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.836632 29845 raft_consensus.cc:2243] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.837258 29845 raft_consensus.cc:2272] T 72d9d9cc8be6411883e9f3f17ef865eb P e9cccf7eb2a545ef8c2169b67ff261bf [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.849417 29845 tablet_replica.cc:333] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.850255 29845 raft_consensus.cc:2243] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.884018 29845 raft_consensus.cc:2272] T 41116db50e8c4b9598abe43f5d36bf12 P e9cccf7eb2a545ef8c2169b67ff261bf [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:01.889999 29845 tablet_replica.cc:333] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf: stopping tablet replica
I20260709 12:27:01.898993 29845 raft_consensus.cc:2243] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:01.899757 29845 raft_consensus.cc:2272] T 7d43e79f1b224452ad15ff525d3a0bd9 P e9cccf7eb2a545ef8c2169b67ff261bf [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:02.109799 31689 consensus_queue.cc:237] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.136855 31068 raft_consensus.cc:1275] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:02.138449  6421 consensus_queue.cc:1048] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:27:02.145072 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:02.159313  6421 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Committing config change with OpId 25.15: config changed from index 3 to 15, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:02.163655 31067 raft_consensus.cc:2955] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Committing config change with OpId 25.15: config changed from index 3 to 15, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:02.175987 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2439e7934b98446ca4e7d82381e8a499 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
I20260709 12:27:02.193475 30884 catalog_manager.cc:5697] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: config changed from index 3 to 15, VOTER ce27a72886964e4287059a1d66bd5b6b (127.29.37.67) evicted. New cstate: current_term: 25 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.330297  6404 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.330886  6404 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.334625  6437 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.335268  6437 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.337399  6437 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:02.346613 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:02.347301 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:02.345367  6404 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:27:02.350908 31109 leader_election.cc:336] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:02.351493 31109 leader_election.cc:336] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.353495 31109 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.354606 31109 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.355135  6404 raft_consensus.cc:2749] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:02.359524  6404 raft_consensus.cc:2749] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:02.361999 31129 tablet_service.cc:1558] Processing DeleteTablet for tablet 2439e7934b98446ca4e7d82381e8a499 with delete_type TABLET_DATA_TOMBSTONED (TS ce27a72886964e4287059a1d66bd5b6b not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:46426
I20260709 12:27:02.382561  6476 tablet_replica.cc:333] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:02.394795  6404 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.395421  6404 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.397487  6404 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:02.403982 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:27:02.414949 31109 leader_election.cc:336] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.415470 31109 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.416216  6404 raft_consensus.cc:2749] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:02.424278  6476 raft_consensus.cc:2243] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:02.425431  6476 raft_consensus.cc:2272] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:02.428861  6476 ts_tablet_manager.cc:1916] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:27:02.449755  6476 ts_tablet_manager.cc:1929] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 25.14
I20260709 12:27:02.450341  6476 log.cc:1199] T 2439e7934b98446ca4e7d82381e8a499 P ce27a72886964e4287059a1d66bd5b6b: Deleting WAL directory at /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/wals/2439e7934b98446ca4e7d82381e8a499
I20260709 12:27:02.452227 30876 catalog_manager.cc:5028] TS ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): tablet 2439e7934b98446ca4e7d82381e8a499 (table test-workload [id=bd8e3f974a454ea9abf75ec8b4c1c3e6]) successfully deleted
I20260709 12:27:02.502345  6335 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.502947  6335 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.505441  6335 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:02.509212 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:02.510643  6431 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.511211  6431 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.513641  6431 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.525502  6421 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.526072  6421 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.528283  6421 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.528792  6421 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.529241  6421 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.531749  6421 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.532400 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:02.533071 31142 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 17.
W20260709 12:27:02.533581 31664 leader_election.cc:336] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.537258 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:02.547961 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:27:02.553364 31033 leader_election.cc:336] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.553932 31033 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.568574  6335 raft_consensus.cc:2749] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:02.568994 31664 leader_election.cc:336] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.582564 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.584810  6335 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.589622  6335 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.587427  6437 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.592335  6437 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.594596  6437 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:27:02.601900 31109 leader_election.cc:336] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.603183  6422 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:02.603701  6422 raft_consensus.cc:493] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.604066  6422 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.610205  6404 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.610841  6404 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.613806  6404 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.619599 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.626175 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:02.636075  6335 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:02.637223  6444 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.640072  6444 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.642756  6444 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:27:02.643460 31033 leader_election.cc:336] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.646728  6465 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:02.653093 31033 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.654100  6437 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.662873  6335 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:02.660607  6437 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.668728 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:02.670849 31697 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 17.
W20260709 12:27:02.659265 31109 leader_election.cc:336] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.671758 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.681360  6465 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.698297 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:02.698879  6465 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.701555  6465 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.702692 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:02.703363 31068 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 22.
I20260709 12:27:02.703401  6422 raft_consensus.cc:515] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.704623  6335 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.705114  6335 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.707194  6335 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.708019  6335 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.708504  6335 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.710453  6335 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:02.716421 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:27:02.717600  6431 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.718184  6431 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.718920  6422 leader_election.cc:290] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.720155  6431 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:02.721201  6422 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:02.721763  6422 raft_consensus.cc:493] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.722201  6422 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:02.723181 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 63952.8us
I20260709 12:27:02.690469  6229 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.745923  6229 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.748147  6229 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.749663 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:02.750332 31068 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 13.
I20260709 12:27:02.751358 31661 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:27:02.752126  6229 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:02.752475  6229 raft_consensus.cc:493] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.752785  6229 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.760298  6229 raft_consensus.cc:515] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.763316 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "363b01e660fa4b36aa4a5112bab1ac09" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:27:02.763875 31068 raft_consensus.cc:3060] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.769716  6465 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.770257  6465 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.772256  6465 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.777151 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:02.777907  6479 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:02.778337  6479 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.778445  6465 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.778764  6479 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.779166  6465 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.781248  6465 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.698076  6437 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.725878  6404 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:02.730376  6422 raft_consensus.cc:515] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.786137  6479 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.792047  6422 leader_election.cc:290] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.794358  6479 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:02.795156 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:27:02.795742 31061 raft_consensus.cc:3060] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:02.799541 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 54065.6us
I20260709 12:27:02.805481  6404 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.806033  6404 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.733961 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:02.735496 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82399103216c4d5e8d2acdbbf3cee84d" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:02.807840 31689 raft_consensus.cc:2393] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate a517aadb0ae04b89a13779d7898dfa0c in this term.
I20260709 12:27:02.738029 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:02.809257 31142 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 24.
I20260709 12:27:02.809999 31058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:02.810619 31058 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:27:02.688509  6444 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.740283 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:02.812789 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:02.813184 31698 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:27:02.744668 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2c77b15766d44f585ab3cb57444a324" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:27:02.813511  6480 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:02.814062  6480 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.814055 31143 raft_consensus.cc:3060] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.814638  6480 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.814934  6404 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.822258  6480 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:27:02.822736 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 45663.6us
I20260709 12:27:02.823182  6444 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.824584 31067 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
I20260709 12:27:02.825191 31061 raft_consensus.cc:2468] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:27:02.827687 31661 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:27:02.827808 31106 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:02.828534  6422 raft_consensus.cc:2804] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:02.828963  6437 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:02.829119  6422 raft_consensus.cc:697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:27:02.830050  6422 consensus_queue.cc:237] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.829437  6437 raft_consensus.cc:493] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.831377  6437 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.823843 31143 raft_consensus.cc:2468] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 18.
I20260709 12:27:02.835772  6480 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.837785 31068 raft_consensus.cc:2468] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 14.
I20260709 12:27:02.838665  6229 leader_election.cc:290] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.839300 31661 leader_election.cc:304] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:27:02.839670 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 100519us
I20260709 12:27:02.838542 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48e42d074cd496f996172cff1a0c835" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:27:02.840464 31061 raft_consensus.cc:3060] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.826694  6444 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.847733  6465 raft_consensus.cc:2804] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:02.849404  6465 raft_consensus.cc:697] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:27:02.850481  6465 consensus_queue.cc:237] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.852893  6444 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.853449  6444 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.855470  6444 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.858297 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:02.858965 31143 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:27:02.859308 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:02.859990 31142 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 23.
I20260709 12:27:02.862569 31061 raft_consensus.cc:2468] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:27:02.867972 31106 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:02.869947  6480 raft_consensus.cc:2804] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:02.870445  6480 raft_consensus.cc:697] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:27:02.871348  6480 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.875653  6465 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.876368  6465 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.880693  6465 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.881517 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:02.882645 31061 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:27:02.883971 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:27:02.885258 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 56772.6us
I20260709 12:27:02.892447  6229 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:02.893011  6229 raft_consensus.cc:493] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.893747  6229 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:02.915759  6422 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.919369 31698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:02.920146  6422 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.918185 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4950edf9dfb9412981a0b325bf866deb" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:27:02.921180 31697 raft_consensus.cc:3060] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.924830  6437 raft_consensus.cc:515] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.930738 31698 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 26.
I20260709 12:27:02.919626 30883 catalog_manager.cc:5697] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 22 to 23, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 23 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.934350  6422 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.934619 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:02.935379  6444 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:02.935717  6444 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.936029  6444 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:02.936529 31697 raft_consensus.cc:2468] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
I20260709 12:27:02.937472 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:02.939311 31061 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 26.
I20260709 12:27:02.940330 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:27:02.941073  6422 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:02.941602  6422 raft_consensus.cc:493] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:02.941962  6422 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:02.943568 30884 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 22 to 23, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:02.950568  6444 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:02.953482  6422 raft_consensus.cc:515] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:02.957338 31109 leader_election.cc:336] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:02.916411 31664 leader_election.cc:336] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.975423 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:02.978916 31142 raft_consensus.cc:2393] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c4511d66b474ee0aa41c9b1f03b66ab in current term 23: Already voted for candidate ce27a72886964e4287059a1d66bd5b6b in this term.
I20260709 12:27:02.978299  6422 leader_election.cc:290] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:02.979641 31143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:27:02.980410 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 64024.3us
W20260709 12:27:02.981179 31664 leader_election.cc:336] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.982525 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544dffe603ca4dce9d6697ea6b15c925" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:27:02.983150 31061 raft_consensus.cc:3060] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:02.983422  6444 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:02.988511 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e9bb54e628458a94d623c051d36c1a" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:27:02.989074 31067 raft_consensus.cc:3060] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.997484 31067 raft_consensus.cc:2468] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:27:02.998426 31106 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:02.998823  6229 raft_consensus.cc:515] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:02.999171 31061 raft_consensus.cc:2468] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 27.
I20260709 12:27:02.999394 31664 leader_election.cc:304] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:27:03.000169 31033 leader_election.cc:336] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.000167 31661 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:27:03.000751 31033 leader_election.cc:336] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.001989  6404 raft_consensus.cc:2804] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:03.002087 31033 leader_election.cc:336] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.005949  6404 raft_consensus.cc:697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
W20260709 12:27:03.006145 31033 leader_election.cc:336] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:02.998602  6437 leader_election.cc:290] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:03.010546  6421 raft_consensus.cc:2804] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:03.011060  6421 raft_consensus.cc:697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 27 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:27:03.011850  6421 consensus_queue.cc:237] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:03.013607  6229 leader_election.cc:290] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:03.006943  6404 consensus_queue.cc:237] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.018425 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83f49f3c3cc44994877e69c3ff83090e" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:27:03.022001 31067 raft_consensus.cc:3060] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.023535  6437 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.024091  6437 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.027006 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:03.020581 31033 leader_election.cc:304] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:03.028759  6478 raft_consensus.cc:2749] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.029950 31033 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: a517aadb0ae04b89a13779d7898dfa0c, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:03.030362  6437 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:03.032231 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:03.033010  6444 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:03.033541  6444 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.033049 30884 catalog_manager.cc:5697] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 13 to 14, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 14 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.033957  6444 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.034972 30884 catalog_manager.cc:5697] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 26 to 27, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 27 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.039904 31033 leader_election.cc:304] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:27:03.039942  6480 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.040724  6480 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.042743  6480 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:03.043756 31068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:03.043777  6335 raft_consensus.cc:2764] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:27:03.048271  6480 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.048871  6480 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.051190  6421 raft_consensus.cc:2804] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:03.051973  6421 raft_consensus.cc:697] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:27:03.052882  6421 consensus_queue.cc:237] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:03.054797 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82382d1e93d244cd8307164a5ac891b0" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:27:03.055429 31139 raft_consensus.cc:3060] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.058702  6480 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:03.062067  6444 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:03.067818  6431 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
W20260709 12:27:03.073872 31033 leader_election.cc:336] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.066552 31061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" is_pre_election: true
I20260709 12:27:03.065949 31139 raft_consensus.cc:2468] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 25.
W20260709 12:27:03.076576 31109 leader_election.cc:336] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.070312 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 70959.9us
I20260709 12:27:03.089581  6431 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.091593  6431 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:03.092474  6444 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:27:03.094203 31109 leader_election.cc:336] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.094985 31109 leader_election.cc:336] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.102490 31061 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 22.
W20260709 12:27:03.102506 31033 leader_election.cc:336] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.085637 31067 raft_consensus.cc:2468] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 25.
I20260709 12:27:03.103425 31033 leader_election.cc:304] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:27:03.103169 31109 leader_election.cc:336] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.103958 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:03.104648 31661 leader_election.cc:304] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
W20260709 12:27:03.105105 31109 leader_election.cc:336] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.105458  6422 raft_consensus.cc:2804] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:03.105602  6444 raft_consensus.cc:2749] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.105729  6480 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:03.106088  6422 raft_consensus.cc:697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:27:03.111171  6422 consensus_queue.cc:237] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:03.104061  6478 raft_consensus.cc:2749] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:03.102532 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:03.118633  6229 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:03.118709 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:27:03.106905 31033 leader_election.cc:336] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.106376  6480 raft_consensus.cc:493] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.105495 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:03.119232  6229 raft_consensus.cc:493] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.119500 31033 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:03.119822  6229 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.123365  6465 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:03.123855 30884 catalog_manager.cc:5697] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 22 to 23, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.124958  6431 raft_consensus.cc:2749] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.123780  6465 raft_consensus.cc:493] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.125814  6465 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.120536  6479 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:03.120182  6480 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.133975 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:03.134584 31139 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 18.
I20260709 12:27:03.134887 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0afdb545bb8f485993259007a00c1841" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:27:03.135627 31697 raft_consensus.cc:3060] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.137782  6229 raft_consensus.cc:515] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:03.145242 31697 raft_consensus.cc:2468] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
W20260709 12:27:03.147433 31033 leader_election.cc:336] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.153792 31033 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:03.155388  6229 leader_election.cc:290] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:03.156757  6444 raft_consensus.cc:2804] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:03.157595  6444 raft_consensus.cc:697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:27:03.158473  6444 consensus_queue.cc:237] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.161396  6465 raft_consensus.cc:515] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:03.156154 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:03.166886  6431 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:03.167245  6431 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.167609  6431 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:03.167877 31109 leader_election.cc:336] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.168301 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:27:03.169250 31109 leader_election.cc:336] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.171815  6479 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:03.177090 30884 catalog_manager.cc:5697] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 17 to 18, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 18 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:03.182760 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 61584.8us
I20260709 12:27:03.174377  6480 raft_consensus.cc:515] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.196866  6465 leader_election.cc:290] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:03.199412  6431 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.200611 30884 catalog_manager.cc:5697] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 24 to 25, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 25 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:03.201843 31109 leader_election.cc:336] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.201409  6431 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:03.205000  6480 leader_election.cc:290] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:03.205971 31067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7f0812b392640ecab5f5257bb0e41ee" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab"
I20260709 12:27:03.206559 31067 raft_consensus.cc:3060] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.213104 31067 raft_consensus.cc:2468] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 23.
I20260709 12:27:03.214494 31106 leader_election.cc:304] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:03.213166 31109 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:03.215350  6404 raft_consensus.cc:2804] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:03.215978  6404 raft_consensus.cc:697] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:27:03.216853  6404 consensus_queue.cc:237] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.215974  6480 raft_consensus.cc:2804] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:03.221613  6480 raft_consensus.cc:697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:27:03.222832  6480 consensus_queue.cc:237] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:03.210634 31664 leader_election.cc:336] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.219096 31033 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c; no voters: 
I20260709 12:27:03.227259  6431 raft_consensus.cc:2804] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:03.228014  6431 raft_consensus.cc:697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:27:03.230114  6431 consensus_queue.cc:237] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:03.226444 31664 leader_election.cc:336] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.252128 31664 leader_election.cc:336] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.258535 31664 leader_election.cc:336] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.264751 31664 leader_election.cc:336] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.260947 31033 leader_election.cc:336] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.269699 31033 leader_election.cc:336] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.266729 30883 catalog_manager.cc:5697] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 22 to 23, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.265121 30884 catalog_manager.cc:5697] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 24 to 25, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 25 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:03.278411 31109 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.282045 29845 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 31013)
I20260709 12:27:03.281926 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ac1b8eaf94e99a3eee16d0ca65113" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:27:03.282662 31139 raft_consensus.cc:3060] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.292699 30880 catalog_manager.cc:5697] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 17 to 18, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 18 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:03.297155 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "635264f51b2d4410a1da9b76606a6350" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:27:03.297761 31135 raft_consensus.cc:3060] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.304592 31135 raft_consensus.cc:2468] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 25.
I20260709 12:27:03.306133 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f57ce098bf541c4b1ca32a9c4b8cec0" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:27:03.306710 31142 raft_consensus.cc:3060] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.308030 31139 raft_consensus.cc:2468] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 19.
I20260709 12:27:03.311756  6335 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.312224  6335 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.314145  6335 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:03.318264 30884 catalog_manager.cc:5697] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 19, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 19 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:03.319173 31142 raft_consensus.cc:2468] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:27:03.328222 31067 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:03.329646  6404 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:27:03.330312 31664 leader_election.cc:336] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.331665 31664 leader_election.cc:336] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.335253 31664 leader_election.cc:336] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.351987 31033 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:03.367616  6335 raft_consensus.cc:2804] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:03.368120  6335 raft_consensus.cc:697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:27:03.369017  6335 consensus_queue.cc:237] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:03.385944 31135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:03.386612 31135 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 14.
I20260709 12:27:03.387306 31664 leader_election.cc:304] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:03.394358  6422 raft_consensus.cc:2804] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:03.414314 31067 raft_consensus.cc:1275] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:27:03.440380  6422 raft_consensus.cc:697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:27:03.462419  6492 consensus_queue.cc:1048] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
W20260709 12:27:03.470539 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 78444.7us
I20260709 12:27:03.471283 31664 leader_election.cc:304] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:03.450579  6422 consensus_queue.cc:237] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:03.477504 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 74991.1us
W20260709 12:27:03.479223 31664 leader_election.cc:336] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.478727 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
W20260709 12:27:03.480824 31664 leader_election.cc:336] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.488813 31664 leader_election.cc:336] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.473650  6503 raft_consensus.cc:2804] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:03.515235 31033 leader_election.cc:336] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.480855  6498 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:03.519550 31061 raft_consensus.cc:1275] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:03.517517  6498 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Starting leader election (detected failure of leader e9cccf7eb2a545ef8c2169b67ff261bf)
I20260709 12:27:03.523172  6498 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:03.536969 31061 raft_consensus.cc:1275] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:03.526599  6503 raft_consensus.cc:697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:27:03.527206  6437 consensus_queue.cc:1048] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:27:03.547199  6503 consensus_queue.cc:237] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:03.549188  6465 consensus_queue.cc:1048] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:03.565953 30884 catalog_manager.cc:5697] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 18 to 19, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 19 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:03.580047 31664 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:03.580873 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.581516 31664 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:03.581804 31059 raft_consensus.cc:1275] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
W20260709 12:27:03.582124 31664 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:03.582742 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:03.596040 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 67025.2us
I20260709 12:27:03.597399 30880 catalog_manager.cc:5697] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 24 to 25, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 25 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:03.607435 31033 leader_election.cc:336] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:03.595196  6465 consensus_queue.cc:1048] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.017s
W20260709 12:27:03.670814 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 49559us
W20260709 12:27:03.683648 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 194444us
I20260709 12:27:03.690263 31060 raft_consensus.cc:1275] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:03.692998  6511 consensus_queue.cc:1048] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:03.708981  6498 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:27:03.716636 31033 consensus_peers.cc:597] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.717486 31139 raft_consensus.cc:1275] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:03.722878 31060 raft_consensus.cc:1275] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:03.733220 31689 raft_consensus.cc:1275] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:03.741289  6465 consensus_queue.cc:1048] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:03.742432  6508 consensus_queue.cc:1048] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:27:03.746265 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.746809 31664 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:03.748734 31142 raft_consensus.cc:1275] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:03.757176  6335 consensus_queue.cc:1048] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:03.766263 30884 catalog_manager.cc:5697] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 23 to 24, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 24 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.764424  6510 consensus_queue.cc:1048] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:27:03.793318  6498 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Requested vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:03.815611 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 59111.9us
I20260709 12:27:03.845322 31139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ea7681ba29145eba9db5bc137cf18b4" candidate_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:27:03.846302 31139 raft_consensus.cc:3060] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:03.867453 31033 consensus_peers.cc:597] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:03.868153 31033 consensus_peers.cc:597] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.868659 31033 consensus_peers.cc:597] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:03.869244 31033 consensus_peers.cc:597] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.869776 31033 consensus_peers.cc:597] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.873080 31135 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:03.887020 31140 raft_consensus.cc:1275] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:03.891867  6504 consensus_queue.cc:1048] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:27:03.893182 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.897496 31689 raft_consensus.cc:1275] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:27:03.910575 31139 raft_consensus.cc:2468] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4511d66b474ee0aa41c9b1f03b66ab in term 15.
I20260709 12:27:03.911638 31033 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:03.908509  6478 consensus_queue.cc:1048] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 12:27:03.925483  6512 raft_consensus.cc:2804] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:03.926076  6512 raft_consensus.cc:697] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Becoming Leader. State: Replica: 2c4511d66b474ee0aa41c9b1f03b66ab, State: Running, Role: LEADER
I20260709 12:27:03.927105  6512 consensus_queue.cc:237] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:27:03.950752 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:03.958019 31033 consensus_peers.cc:597] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.958961 31033 leader_election.cc:336] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:03.963023 31033 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.969846 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.969889 31143 raft_consensus.cc:1275] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:27:03.974617 31033 consensus_peers.cc:597] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.975217 31033 consensus_peers.cc:597] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:03.976080 31033 consensus_peers.cc:597] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:03.979379  6478 consensus_queue.cc:1048] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:03.995113 31033 consensus_peers.cc:597] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:03.995970 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd024d82eb9c473ba31252aeeb287fc0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:04.004643 31067 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:04.006444  6508 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:27:04.090968 31033 consensus_peers.cc:597] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:04.100978  6510 raft_consensus.cc:2955] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Committing config change with OpId 25.18: config changed from index -1 to 18, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:27:04.120301 30883 catalog_manager.cc:5697] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: term changed from 14 to 15, leader changed from e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) to 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66). New cstate: current_term: 15 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:04.180169 31033 consensus_peers.cc:597] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.183032 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.188779 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:04.199388 31143 raft_consensus.cc:1275] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:04.126672 31064 raft_consensus.cc:2955] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Committing config change with OpId 25.18: config changed from index -1 to 18, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:27:04.201895  6422 consensus_queue.cc:1048] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:04.230309 31143 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
W20260709 12:27:04.253525 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.278982 31060 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:04.281019  6480 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:27:04.282030 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.284042 31109 consensus_peers.cc:597] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 676d422dffb24e3cb16409a80bb28758. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:04.284504 29845 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:04.369149 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.386557 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 62584.3us
I20260709 12:27:04.418814  6480 raft_consensus.cc:2955] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Committing config change with OpId 23.14: config changed from index -1 to 14, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:27:04.428387  6544 ts_tablet_manager.cc:933] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:04.437670  6544 tablet_copy_client.cc:323] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:27:04.478962 30884 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 18, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 25 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.523336 31060 raft_consensus.cc:2955] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index -1 to 14, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:27:04.518282 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet dd024d82eb9c473ba31252aeeb287fc0 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:27:04.536571 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet dd024d82eb9c473ba31252aeeb287fc0 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-dd024d82eb9c473ba31252aeeb287fc0
W20260709 12:27:04.539029 31033 consensus_peers.cc:597] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:04.564769 31140 raft_consensus.cc:1275] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:04.602890  6502 consensus_queue.cc:1048] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
W20260709 12:27:04.611186 31664 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:04.628456 30884 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 14, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.647063 31154 tablet_copy_source_session.cc:215] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:04.651858  6544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd024d82eb9c473ba31252aeeb287fc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:04.639938 31064 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
W20260709 12:27:04.694538 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 61037.1us
I20260709 12:27:04.721473  6544 tablet_copy_client.cc:806] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:04.722318  6544 tablet_copy_client.cc:670] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:04.729012  6544 tablet_copy_client.cc:538] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:04.768182 31109 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.769023 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:04.769675 31109 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.776871 31107 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.779383 31139 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:04.785794  6514 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:27:04.789215 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:04.808941 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e98467ef2b2b4bab954b704025d182b4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:04.814559  6550 ts_tablet_manager.cc:933] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:04.818303  6512 raft_consensus.cc:2955] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 LEADER]: Committing config change with OpId 27.19: config changed from index -1 to 19, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:27:04.830322 31139 raft_consensus.cc:2955] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Committing config change with OpId 27.19: config changed from index -1 to 19, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:27:04.835183  6544 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:27:04.847875  6550 tablet_copy_client.cc:323] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
W20260709 12:27:04.849095 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.849668 31109 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.850158 31109 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.852007 31109 consensus_peers.cc:597] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:04.853699 31109 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.864042 31109 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:04.865926 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet 676d422dffb24e3cb16409a80bb28758 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:27:04.866564 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet 676d422dffb24e3cb16409a80bb28758 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-676d422dffb24e3cb16409a80bb28758
I20260709 12:27:04.872164 30884 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index -1 to 19, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 27 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:27:04.864943 31109 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.893841 31109 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:04.894590 31109 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:04.895139 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:04.899240 31109 consensus_peers.cc:597] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:04.908601 31135 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:04.996840 31154 tablet_copy_source_session.cc:215] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:27:05.035425 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 68019.1us
W20260709 12:27:05.040036 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 53072.1us
W20260709 12:27:05.042279 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 74102.6us
I20260709 12:27:05.079190  6550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676d422dffb24e3cb16409a80bb28758. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:05.086952 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:05.105435 31060 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:05.121593  6510 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:05.141386 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:05.157043 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417): Couldn't send request to peer a517aadb0ae04b89a13779d7898dfa0c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d48e42d074cd496f996172cff1a0c835. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:05.187269  6562 ts_tablet_manager.cc:933] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:05.187316  6480 raft_consensus.cc:2955] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Committing config change with OpId 23.13: config changed from index -1 to 13, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:27:05.190176 31064 raft_consensus.cc:2955] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Committing config change with OpId 23.13: config changed from index -1 to 13, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
W20260709 12:27:05.184394 31109 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:05.205528  6562 tablet_copy_client.cc:323] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.66:45157
I20260709 12:27:05.207943 31078 tablet_copy_service.cc:140] P 2c4511d66b474ee0aa41c9b1f03b66ab: Received BeginTabletCopySession request for tablet e98467ef2b2b4bab954b704025d182b4 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:38956)
I20260709 12:27:05.208565 31078 tablet_copy_service.cc:161] P 2c4511d66b474ee0aa41c9b1f03b66ab: Beginning new tablet copy session on tablet e98467ef2b2b4bab954b704025d182b4 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:38956: session id = a517aadb0ae04b89a13779d7898dfa0c-e98467ef2b2b4bab954b704025d182b4
I20260709 12:27:05.210520 30883 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab reported cstate change: config changed from index -1 to 13, NON_VOTER a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) added. New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } } }
I20260709 12:27:05.253556  6544 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:05.254945  6544 tablet_bootstrap.cc:492] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:27:05.255043 31078 tablet_copy_source_session.cc:215] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:05.255854  6544 ts_tablet_manager.cc:1403] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.421s	user 0.057s	sys 0.017s
I20260709 12:27:05.256642  6550 tablet_copy_client.cc:806] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:05.257227  6550 tablet_copy_client.cc:670] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:05.264124  6562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e98467ef2b2b4bab954b704025d182b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:05.277338  6550 tablet_copy_client.cc:538] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:05.285353 29845 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 31013)
I20260709 12:27:05.296514  6562 tablet_copy_client.cc:806] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:05.297659  6550 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:27:05.297660  6562 tablet_copy_client.cc:670] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:05.318390  6562 tablet_copy_client.cc:538] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:27:05.326858  6544 raft_consensus.cc:359] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:05.342387  6544 raft_consensus.cc:740] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:27:05.343927  6544 consensus_queue.cc:260] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:05.391317  6544 ts_tablet_manager.cc:1434] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.135s	user 0.006s	sys 0.002s
I20260709 12:27:05.411917 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-dd024d82eb9c473ba31252aeeb287fc0 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:27:05.412458 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-dd024d82eb9c473ba31252aeeb287fc0 on tablet dd024d82eb9c473ba31252aeeb287fc0 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:27:05.423620  6562 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
I20260709 12:27:05.489852  6550 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:05.499533  6550 tablet_bootstrap.cc:492] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:27:05.502490  6550 ts_tablet_manager.cc:1403] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.205s	user 0.043s	sys 0.006s
I20260709 12:27:05.514360  6550 raft_consensus.cc:359] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:05.521327  6550 raft_consensus.cc:740] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:27:05.522084  6550 consensus_queue.cc:260] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:05.525205  6550 ts_tablet_manager.cc:1434] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.003s
I20260709 12:27:05.545121 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-676d422dffb24e3cb16409a80bb28758 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:27:05.545563 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-676d422dffb24e3cb16409a80bb28758 on tablet 676d422dffb24e3cb16409a80bb28758 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:27:05.658856  6550 ts_tablet_manager.cc:933] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: Initiating tablet copy from peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:05.676890  6550 tablet_copy_client.cc:323] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Beginning tablet copy session from remote peer at address 127.29.37.67:36565
I20260709 12:27:05.678534 31154 tablet_copy_service.cc:140] P ce27a72886964e4287059a1d66bd5b6b: Received BeginTabletCopySession request for tablet d48e42d074cd496f996172cff1a0c835 from peer a517aadb0ae04b89a13779d7898dfa0c ({username='slave'} at 127.0.0.1:42006)
I20260709 12:27:05.679047 31154 tablet_copy_service.cc:161] P ce27a72886964e4287059a1d66bd5b6b: Beginning new tablet copy session on tablet d48e42d074cd496f996172cff1a0c835 from peer a517aadb0ae04b89a13779d7898dfa0c at {username='slave'} at 127.0.0.1:42006: session id = a517aadb0ae04b89a13779d7898dfa0c-d48e42d074cd496f996172cff1a0c835
I20260709 12:27:05.694837 31154 tablet_copy_source_session.cc:215] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:05.737422  6550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d48e42d074cd496f996172cff1a0c835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:05.740430 31689 raft_consensus.cc:1217] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEARNER]: Deduplicated request from leader. Original: 23.13->[23.14-23.14]   Dedup: 23.14->[]
I20260709 12:27:05.746466  6562 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:05.747732  6562 tablet_bootstrap.cc:492] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:27:05.757586  6562 ts_tablet_manager.cc:1403] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.334s	user 0.061s	sys 0.004s
I20260709 12:27:05.762907  6562 raft_consensus.cc:359] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:05.774892  6562 raft_consensus.cc:740] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:27:05.786288  6562 consensus_queue.cc:260] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:05.802294  6562 ts_tablet_manager.cc:1434] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:27:05.804680 31078 tablet_copy_service.cc:342] P 2c4511d66b474ee0aa41c9b1f03b66ab: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-e98467ef2b2b4bab954b704025d182b4 received from {username='slave'} at 127.0.0.1:38956
I20260709 12:27:05.805158 31078 tablet_copy_service.cc:434] P 2c4511d66b474ee0aa41c9b1f03b66ab: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-e98467ef2b2b4bab954b704025d182b4 on tablet e98467ef2b2b4bab954b704025d182b4 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:27:05.818722  6550 tablet_copy_client.cc:806] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:05.819306  6550 tablet_copy_client.cc:670] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:05.846490 31689 raft_consensus.cc:1217] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEARNER]: Deduplicated request from leader. Original: 25.17->[25.18-25.18]   Dedup: 25.18->[]
I20260709 12:27:05.855635  6550 tablet_copy_client.cc:538] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:05.870924 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:05.891237 31664 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:05.896258  6550 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap starting.
W20260709 12:27:05.898069 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:05.972483 31664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111) [suppressed 402 similar messages]
W20260709 12:27:05.973222 31664 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:06.028019 31033 consensus_peers.cc:597] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.062299  6550 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:06.063360  6550 tablet_bootstrap.cc:492] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: Bootstrap complete.
I20260709 12:27:06.064145  6550 ts_tablet_manager.cc:1403] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: Time spent bootstrapping tablet: real 0.168s	user 0.035s	sys 0.009s
W20260709 12:27:06.067350 31664 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:06.075074 31664 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:06.075613 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.079269  6550 raft_consensus.cc:359] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:06.092198  6550 raft_consensus.cc:740] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Initialized, Role: LEARNER
I20260709 12:27:06.092854  6550 consensus_queue.cc:260] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: NON_VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: true } }
I20260709 12:27:06.101434  6550 ts_tablet_manager.cc:1434] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:27:06.114612 31154 tablet_copy_service.cc:342] P ce27a72886964e4287059a1d66bd5b6b: Request end of tablet copy session a517aadb0ae04b89a13779d7898dfa0c-d48e42d074cd496f996172cff1a0c835 received from {username='slave'} at 127.0.0.1:42006
I20260709 12:27:06.115042 31154 tablet_copy_service.cc:434] P ce27a72886964e4287059a1d66bd5b6b: ending tablet copy session a517aadb0ae04b89a13779d7898dfa0c-d48e42d074cd496f996172cff1a0c835 on tablet d48e42d074cd496f996172cff1a0c835 with peer a517aadb0ae04b89a13779d7898dfa0c
I20260709 12:27:06.179867  6510 raft_consensus.cc:1064] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
W20260709 12:27:06.189059 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.192328  6510 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 12, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:06.228263 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:06.245594 31033 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.246457 31033 consensus_peers.cc:597] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:06.249079 31060 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:06.251287  6571 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:06.276369 31698 raft_consensus.cc:1217] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEARNER]: Deduplicated request from leader. Original: 23.12->[23.13-23.13]   Dedup: 23.13->[]
I20260709 12:27:06.277338 31689 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:27:06.280750 31033 consensus_peers.cc:597] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.285835 29845 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:06.323387 31033 consensus_peers.cc:597] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.327303 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51347.6us
W20260709 12:27:06.351006 31033 consensus_peers.cc:597] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.353266 31033 consensus_peers.cc:597] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.360220  6571 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:06.366401  6570 raft_consensus.cc:1064] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:27:06.369068  6570 consensus_queue.cc:237] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 16, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:06.376255 31033 consensus_peers.cc:597] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.381063 31109 consensus_peers.cc:597] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:06.392858 31696 raft_consensus.cc:1217] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 LEARNER]: Deduplicated request from leader. Original: 27.18->[27.19-27.19]   Dedup: 27.19->[]
I20260709 12:27:06.396502 31064 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:06.400128  6570 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:27:06.422106 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.447672 31033 consensus_peers.cc:597] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.450311  6574 raft_consensus.cc:2955] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index 14 to 15, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.451874 31697 raft_consensus.cc:1275] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:06.467252 31064 raft_consensus.cc:2955] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index 14 to 15, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.475258 30883 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 14 to 15, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:06.481065 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.491564  6571 consensus_queue.cc:1048] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:27:06.498018 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:06.494956 31697 raft_consensus.cc:2955] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index 14 to 15, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } attrs { replace: true } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:27:06.531937 31033 consensus_peers.cc:597] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:06.545249 31033 consensus_peers.cc:597] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:06.545928 31033 consensus_peers.cc:597] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.548156  6570 raft_consensus.cc:2955] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Committing config change with OpId 25.19: config changed from index 18 to 19, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.550834 31689 raft_consensus.cc:2955] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Committing config change with OpId 25.19: config changed from index 18 to 19, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.599614 31064 raft_consensus.cc:2955] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Committing config change with OpId 25.19: config changed from index 18 to 19, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.653235 31139 consensus_queue.cc:237] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:06.666467 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:06.670544 30883 catalog_manager.cc:5697] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 18 to 19, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 25 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:06.697253 31060 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:06.703483  6480 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:06.712224  6571 raft_consensus.cc:2955] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.714811 31697 raft_consensus.cc:1275] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:06.720635 31064 raft_consensus.cc:2955] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.731866  6480 raft_consensus.cc:1064] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:27:06.735060  6480 consensus_queue.cc:237] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 11, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:06.735663  6570 consensus_queue.cc:1048] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:27:06.742575 31109 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.745023 31697 raft_consensus.cc:2955] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.762125 31060 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:06.763402 30876 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 676d422dffb24e3cb16409a80bb28758 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:27:06.770550  6480 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:06.770465 30884 catalog_manager.cc:5697] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 15 to 16, VOTER e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65) evicted. New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:06.783416 31689 raft_consensus.cc:1275] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEARNER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:06.792007  6571 consensus_queue.cc:1048] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
W20260709 12:27:06.792029 31109 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.802538 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:06.804852 31033 consensus_peers.cc:597] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.825712 31109 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:06.828135 30876 catalog_manager.cc:4755] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676d422dffb24e3cb16409a80bb28758: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:06.861547  6570 raft_consensus.cc:2955] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Committing config change with OpId 23.14: config changed from index 13 to 14, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.866040 31697 raft_consensus.cc:2955] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.884262 31060 raft_consensus.cc:2955] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:06.891364  6582 raft_consensus.cc:1064] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: attempting to promote NON_VOTER a517aadb0ae04b89a13779d7898dfa0c to VOTER
I20260709 12:27:06.894788  6582 consensus_queue.cc:237] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 17, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:06.898147 30884 catalog_manager.cc:5697] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 13 to 14, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:06.912985 31033 consensus_peers.cc:597] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:06.960605 31135 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:06.961661 31697 raft_consensus.cc:1275] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 LEARNER]: Refusing update from remote peer 2c4511d66b474ee0aa41c9b1f03b66ab: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:06.963850  6581 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:06.967178  6581 consensus_queue.cc:1048] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:27:06.971400 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:06.979584 31033 consensus_peers.cc:597] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:06.988171 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.034452 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.036216 31109 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:07.044512  6512 raft_consensus.cc:2955] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 LEADER]: Committing config change with OpId 27.20: config changed from index 19 to 20, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:07.056624 31135 raft_consensus.cc:2955] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Committing config change with OpId 27.20: config changed from index 19 to 20, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:27:07.070577 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:07.074959 30880 catalog_manager.cc:5697] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: config changed from index 19 to 20, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
I20260709 12:27:07.080734 31697 raft_consensus.cc:2955] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Committing config change with OpId 27.20: config changed from index 19 to 20, a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } } }
W20260709 12:27:07.103376 31109 consensus_peers.cc:597] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:07.105902 31109 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.184139 31109 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.221015 31109 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.272943 31109 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.286870 29845 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:07.305075 31109 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:07.327112 31109 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.343437 31664 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:07.638685 31109 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:08.076874 30876 catalog_manager.cc:4755] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676d422dffb24e3cb16409a80bb28758: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:08.287340 29845 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:08.287109 31664 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:08.396016 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.435568 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:08.446280 31033 consensus_peers.cc:597] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.543596 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.548375 31664 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:08.560729 31033 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.565413 31664 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:08.643343 31033 consensus_peers.cc:597] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.673524 31033 consensus_peers.cc:597] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.708076 31664 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:08.783618 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.788594 31033 consensus_peers.cc:597] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.796329 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:08.796859 31033 consensus_peers.cc:597] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.864917 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.874542 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:08.943485 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:08.986673 31033 consensus_peers.cc:597] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.005959 31033 consensus_peers.cc:597] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:09.008191 31033 consensus_peers.cc:597] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.077607 31109 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.113874 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:09.177754 30876 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 44212.7us
W20260709 12:27:09.181430 30876 catalog_manager.cc:4755] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676d422dffb24e3cb16409a80bb28758: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:09.214898 31033 consensus_peers.cc:597] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:09.215483 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:09.220292 31033 consensus_peers.cc:597] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:09.224288 31109 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.234560 31109 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.235150 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:36565, user_credentials={real_user=slave}} blocked reactor thread for 51846.4us
W20260709 12:27:09.288082 29845 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:09.295519 31033 consensus_peers.cc:597] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.315662 31033 consensus_peers.cc:597] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.443663 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.489166 31109 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.525048 31109 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:09.529006 31109 consensus_peers.cc:597] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:09.565883 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:09.576918 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.613677 31033 consensus_peers.cc:597] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:09.614310 31033 consensus_peers.cc:597] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.665431 31109 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.677441 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.680848 31109 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.862403 31109 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.919770 31664 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:09.946442 31109 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:09.950294 31109 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:10.124110 31109 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:10.289790 29845 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:10.749496 31664 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:10.882172 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:10.960071 31664 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:10.977777 31109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111) [suppressed 461 similar messages]
W20260709 12:27:11.013178 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:11.031816 31033 consensus_peers.cc:597] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.093449 31033 consensus_peers.cc:597] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.160508 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.169746 31033 consensus_peers.cc:597] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.178117 31664 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:11.178843 31664 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:11.209950 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.289739 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:11.290333 29845 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:11.343207 31033 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.368773 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 98064.3us
W20260709 12:27:11.458724 30876 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 198425us
W20260709 12:27:11.467450 30876 catalog_manager.cc:4755] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676d422dffb24e3cb16409a80bb28758: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:11.506570 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:11.520341 31033 consensus_peers.cc:597] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.521024 31033 consensus_peers.cc:597] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.674405 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.676962 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.736768 31109 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.738417 31109 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.739725 31109 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.815320 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:11.843693 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 125611us
W20260709 12:27:11.846095 31033 consensus_peers.cc:597] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:11.859838 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:11.864122 31109 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.884996 31033 consensus_peers.cc:597] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.926576 31033 consensus_peers.cc:597] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.927201 31033 consensus_peers.cc:597] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:11.929510 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:11.954612 31033 consensus_peers.cc:597] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:11.955238 31033 consensus_peers.cc:597] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.056700 31033 consensus_peers.cc:597] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.103050 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.121716 31109 consensus_peers.cc:597] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:12.190481 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.202869 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:12.231653 31664 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.290814 29845 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:12.295701 31109 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.333945 31109 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:12.357298 31033 consensus_peers.cc:597] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:12.374732 31033 consensus_peers.cc:597] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.387614 31109 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.413062 31109 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.544943 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08003e1d60 after lost signal to thread 29848
W20260709 12:27:12.546144 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800400140 after lost signal to thread 30924
W20260709 12:27:12.547632 30921 debug-util.cc:398] Leaking SignalData structure 0x7b0800104220 after lost signal to thread 31093
W20260709 12:27:12.548830 30921 debug-util.cc:398] Leaking SignalData structure 0x7b08003bce60 after lost signal to thread 31168
W20260709 12:27:12.550066 30921 debug-util.cc:398] Leaking SignalData structure 0x7b080023f460 after lost signal to thread 31722
W20260709 12:27:12.669054 31109 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:12.866673 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 174201us
W20260709 12:27:12.936849 31109 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:13.007212 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 70418.7us
W20260709 12:27:13.037549 31109 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:13.291303 29845 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:13.327507 31664 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:13.554998 31664 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:13.564227 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.628626 31664 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:13.639452 31033 consensus_peers.cc:597] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.648746 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.671442 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.794179 31033 consensus_peers.cc:597] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.818185 31033 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.818709 31033 consensus_peers.cc:597] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.820501 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:13.830979 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08001f22e0 after lost signal to thread 29848
W20260709 12:27:13.832302 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9e20 after lost signal to thread 30924
W20260709 12:27:13.833865 31165 debug-util.cc:398] Leaking SignalData structure 0x7b0800354d40 after lost signal to thread 31093
W20260709 12:27:13.841881 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08002316a0 after lost signal to thread 31168
W20260709 12:27:13.844427 31165 debug-util.cc:398] Leaking SignalData structure 0x7b08001cc580 after lost signal to thread 31722
W20260709 12:27:13.861359 31664 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:13.888233 31109 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.899386 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:13.953181 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 52678.5us
W20260709 12:27:13.956390 31033 consensus_peers.cc:597] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.980857 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:13.982808 31033 consensus_peers.cc:597] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:13.988684 31033 consensus_peers.cc:597] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.000226 31033 consensus_peers.cc:597] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.010303 31109 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.125025 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.248278 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:14.292037 29845 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:14.294060 31109 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.308383 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.353006 31033 consensus_peers.cc:597] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:14.464016 31033 consensus_peers.cc:597] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:14.473755 31033 consensus_peers.cc:597] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.487958 31033 consensus_peers.cc:597] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.579798 31109 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.580777 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:14.732874 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.761842 31033 consensus_peers.cc:597] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:14.775962 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:14.790912 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.819250 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.858211 31033 consensus_peers.cc:597] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:14.860996 31664 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.868867 31109 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:14.882421 31109 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.883025 31109 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.885842 31033 consensus_peers.cc:597] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.956626 31109 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:14.972692 31109 consensus_peers.cc:597] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:15.133638 31109 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:15.292692 29845 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 31013)
W20260709 12:27:15.485036 31109 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:15.630909 30876 catalog_manager.cc:4755] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676d422dffb24e3cb16409a80bb28758: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:15.673501 31109 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:15.697777 31664 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:15.846596 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.881070 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:15.980177 31109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111) [suppressed 451 similar messages]
W20260709 12:27:16.159994 31033 consensus_peers.cc:597] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.189743 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.249186 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:16.250196 31664 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:16.289603 31106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 86696.5us
W20260709 12:27:16.302699 31033 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 141998us
W20260709 12:27:16.334969 31109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:35417, user_credentials={real_user=slave}} blocked reactor thread for 124008us
W20260709 12:27:16.335479 31033 consensus_peers.cc:597] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.371438 31664 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:16.382198 31664 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:16.772675 31109 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.783298 31033 consensus_peers.cc:597] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.791219 31033 consensus_peers.cc:597] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.796490 31033 consensus_peers.cc:597] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.796979 31033 consensus_peers.cc:597] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.797420 31033 consensus_peers.cc:597] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.798669 31033 consensus_peers.cc:597] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.798211 31109 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.817270 31033 consensus_peers.cc:597] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:16.829535 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:16.855479 31109 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.857668 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:16.875077 31033 consensus_peers.cc:597] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:16.921697 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.943610 31033 consensus_peers.cc:597] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:16.995118 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:17.091454 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:17.100728 31109 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.143743 31033 consensus_peers.cc:597] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.166685 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.253628 31033 consensus_peers.cc:597] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:17.272353 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.300623 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.305168 31033 consensus_peers.cc:597] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.310673 31033 consensus_peers.cc:597] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:17.339035 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:17.414791 29845 tablet_server.cc:196] TabletServer@127.29.37.65:0 shutdown complete.
W20260709 12:27:17.422178 31090 debug-util.cc:398] Leaking SignalData structure 0x7b080033a220 after lost signal to thread 29848
W20260709 12:27:17.423666 31090 debug-util.cc:398] Leaking SignalData structure 0x7b0800347f60 after lost signal to thread 30924
W20260709 12:27:17.425392 31090 debug-util.cc:398] Leaking SignalData structure 0x7b08001bbb00 after lost signal to thread 31093
W20260709 12:27:17.426869 31090 debug-util.cc:398] Leaking SignalData structure 0x7b080016da80 after lost signal to thread 31168
W20260709 12:27:17.428382 31090 debug-util.cc:398] Leaking SignalData structure 0x7b0800302f20 after lost signal to thread 31722
W20260709 12:27:17.440412 31033 consensus_peers.cc:597] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:17.459137 31109 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.470486 31109 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:17.554989 31109 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:17.598184 29845 tablet_server.cc:179] TabletServer@127.29.37.66:0 shutting down...
W20260709 12:27:17.632043 31106 consensus_peers.cc:597] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.675139 31109 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.679271 31109 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.686389 31106 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.698947 31106 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.716375 31661 consensus_peers.cc:597] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.720769 31661 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.766325 31106 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.785825 31106 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.786659 31106 consensus_peers.cc:597] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.794121 31106 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.807135 31106 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:17.816658 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:27:17.817608 31664 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:17.815990 31106 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:17.834616 29845 tablet_replica.cc:333] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:17.835748 29845 raft_consensus.cc:2243] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:17.836819 29845 raft_consensus.cc:2272] T 0ec17bf7d52c4cb69f4d488858fecc8b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:17.848086 31109 consensus_peers.cc:597] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:17.849951 29845 tablet_replica.cc:333] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:17.851115 29845 raft_consensus.cc:2243] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:17.851797 29845 raft_consensus.cc:2272] T d521786b7ccd41da81f7fd0d175bf582 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:17.855921 31661 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.856586 31661 consensus_peers.cc:597] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:17.857085 29845 tablet_replica.cc:333] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
W20260709 12:27:17.849982 31106 consensus_peers.cc:597] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.867285 31106 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.867880 31106 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.857437 31661 consensus_peers.cc:597] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:17.868666 31661 consensus_peers.cc:597] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:17.873201 29845 raft_consensus.cc:2243] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:27:17.875341 31661 consensus_peers.cc:597] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:17.895339 29845 raft_consensus.cc:2272] T 76399531471a4790a56d52a6ea052864 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:17.911724 29845 tablet_replica.cc:333] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:17.912710 29845 raft_consensus.cc:2243] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:17.914860 31109 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:17.929337 31661 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:17.913319 29845 raft_consensus.cc:2272] T 95980265645c4f7a8d6b201f0ae58174 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:17.934605 29845 tablet_replica.cc:333] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:17.943163 29845 raft_consensus.cc:2243] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:17.944428 29845 raft_consensus.cc:2272] T 1aff3791e3404a7e92be14f04f9674ec P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:17.947376 29845 tablet_replica.cc:333] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:17.954978 29845 raft_consensus.cc:2243] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:17.956141 29845 raft_consensus.cc:2272] T f311574d640f4ff69dd8e9525a8fe8c1 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:17.959092 29845 tablet_replica.cc:333] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:17.967206 29845 raft_consensus.cc:2243] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:17.968053 29845 raft_consensus.cc:2272] T 4a3e0211fca8429086a4403f1ec42a59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:17.971290 29845 tablet_replica.cc:333] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:17.972144 29845 raft_consensus.cc:2243] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Raft consensus shutting down.
W20260709 12:27:17.977358 31661 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:17.978822 29845 raft_consensus.cc:2272] T 0afdb545bb8f485993259007a00c1841 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:18.001406 31661 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:18.005825 31661 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:18.007143 31661 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:18.008633 29845 tablet_replica.cc:333] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.009701 29845 raft_consensus.cc:2243] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.017709 29845 raft_consensus.cc:2272] T 676d422dffb24e3cb16409a80bb28758 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:18.023512 31661 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:18.023963 29845 tablet_replica.cc:333] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.024850 29845 raft_consensus.cc:2243] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.025492 29845 raft_consensus.cc:2272] T 9bb92178abcc4ef385f771bd04448a5b P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.028549 29845 tablet_replica.cc:333] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.033900 29845 raft_consensus.cc:2243] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.037457 29845 raft_consensus.cc:2272] T d8e9bb54e628458a94d623c051d36c1a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:18.042435 31106 consensus_peers.cc:597] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:18.051862 29845 tablet_replica.cc:333] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.055897 29845 raft_consensus.cc:2243] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.058959 29845 raft_consensus.cc:2272] T 9dab2cd673924dc2a2f8622e76701947 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:18.059928 31106 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:18.060395 31109 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:18.073628 29845 tablet_replica.cc:333] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.074502 29845 raft_consensus.cc:2243] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.075476 29845 raft_consensus.cc:2272] T 82399103216c4d5e8d2acdbbf3cee84d P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.080210 29845 tablet_replica.cc:333] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.081082 29845 raft_consensus.cc:2243] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.082834 29845 raft_consensus.cc:2272] T ff2c202e27d24d1baad8ddf5d9db4391 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.091691 29845 tablet_replica.cc:333] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.097206 29845 raft_consensus.cc:2243] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.101102 29845 raft_consensus.cc:2272] T 330f8038ee4e498ebefd9d773f4e7199 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.104960 29845 tablet_replica.cc:333] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.109460 29845 raft_consensus.cc:2243] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.110562 29845 raft_consensus.cc:2272] T d48e42d074cd496f996172cff1a0c835 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.116524 29845 tablet_replica.cc:333] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.117257 29845 raft_consensus.cc:2243] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.119331 29845 raft_consensus.cc:2272] T c6dbea78f5c04486a0fc3f17e2260e2c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.124622 29845 tablet_replica.cc:333] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.128203 29845 raft_consensus.cc:2243] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.130116 29845 raft_consensus.cc:2272] T 83f49f3c3cc44994877e69c3ff83090e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.133002 29845 tablet_replica.cc:333] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
W20260709 12:27:18.135720 31661 consensus_peers.cc:597] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:18.139465 29845 raft_consensus.cc:2243] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.140966 29845 raft_consensus.cc:2272] T 0c6a2f53447447459d88ab978179c65c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.145160 29845 tablet_replica.cc:333] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.161734 29845 raft_consensus.cc:2243] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.169800 29845 raft_consensus.cc:2272] T 39b66c047b4e42afad80738552eee0dc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.179863 29845 tablet_replica.cc:333] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.180738 29845 raft_consensus.cc:2243] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.182039 29845 raft_consensus.cc:2272] T 5bd1ada4571045f6ba305e207cccddcf P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.191129 29845 tablet_replica.cc:333] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.192498 29845 raft_consensus.cc:2243] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.201711 29845 raft_consensus.cc:2272] T f93e9d74965c45a19eb84bff3d1f9825 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.204576 29845 tablet_replica.cc:333] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.205384 29845 raft_consensus.cc:2243] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.206055 29845 raft_consensus.cc:2272] T 13959daae3c149d6ae8248d1f3ca6e59 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.212352 29845 tablet_replica.cc:333] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.213572 29845 raft_consensus.cc:2243] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.219126 29845 raft_consensus.cc:2272] T e98467ef2b2b4bab954b704025d182b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.225939 29845 tablet_replica.cc:333] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.234835 29845 raft_consensus.cc:2243] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.235543 29845 raft_consensus.cc:2272] T a7f0812b392640ecab5f5257bb0e41ee P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.238818 29845 tablet_replica.cc:333] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.241127 29845 raft_consensus.cc:2243] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.244118 29845 raft_consensus.cc:2272] T 82382d1e93d244cd8307164a5ac891b0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.248433 29845 tablet_replica.cc:333] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.256477 29845 raft_consensus.cc:2243] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.260248 29845 raft_consensus.cc:2272] T 4d522233ee4c4536b4fc944d7866c4d9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.262984 29845 tablet_replica.cc:333] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.270618 29845 raft_consensus.cc:2243] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.271284 29845 raft_consensus.cc:2272] T 4ff17d7bb34443adb04e437ed0079e51 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.273774 29845 tablet_replica.cc:333] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.274520 29845 raft_consensus.cc:2243] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.275242 29845 raft_consensus.cc:2272] T 48047b2b139146c38a821328cf7bd148 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.287346 29845 tablet_replica.cc:333] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.289088 29845 raft_consensus.cc:2243] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.291910 29845 raft_consensus.cc:2272] T dd024d82eb9c473ba31252aeeb287fc0 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.300077 29845 tablet_replica.cc:333] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.300802 29845 raft_consensus.cc:2243] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.301558 29845 raft_consensus.cc:2272] T 544dffe603ca4dce9d6697ea6b15c925 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.306492 29845 tablet_replica.cc:333] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.314399 29845 raft_consensus.cc:2243] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.315824 29845 raft_consensus.cc:2272] T bc9ac1b8eaf94e99a3eee16d0ca65113 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.318599 29845 tablet_replica.cc:333] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.319373 29845 raft_consensus.cc:2243] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.320468 29845 raft_consensus.cc:2272] T e39858333670419fb20dc8adde042dd2 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:18.325030 31664 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:18.328351 29845 tablet_replica.cc:333] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.330868 29845 raft_consensus.cc:2243] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.337393 29845 raft_consensus.cc:2272] T 8e81e4942d094afda44355ba1d9c7735 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.344683 29845 tablet_replica.cc:333] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.345541 29845 raft_consensus.cc:2243] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.346140 29845 raft_consensus.cc:2272] T 7d43e79f1b224452ad15ff525d3a0bd9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.348785 29845 tablet_replica.cc:333] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.349596 29845 raft_consensus.cc:2243] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.350802 29845 raft_consensus.cc:2272] T b4162f35b54e4acc8abf0a1f9fc084a9 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.353556 29845 tablet_replica.cc:333] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.354293 29845 raft_consensus.cc:2243] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.361891 29845 raft_consensus.cc:2272] T 7ea7681ba29145eba9db5bc137cf18b4 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.364733 29845 tablet_replica.cc:333] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.365460 29845 raft_consensus.cc:2243] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.370582 29845 raft_consensus.cc:2272] T b1c1f8d5241540fdb4fb4daa1691260f P 2c4511d66b474ee0aa41c9b1f03b66ab [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.381125 29845 tablet_replica.cc:333] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.381870 29845 raft_consensus.cc:2243] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.382550 29845 raft_consensus.cc:2272] T 05394269d1a4461bb79d6f93ab25ff4e P 2c4511d66b474ee0aa41c9b1f03b66ab [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.385422 29845 tablet_replica.cc:333] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.390196 29845 raft_consensus.cc:2243] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.391302 29845 raft_consensus.cc:2272] T 924fd2989ab145f09d211890a85a03fc P 2c4511d66b474ee0aa41c9b1f03b66ab [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.393874 29845 tablet_replica.cc:333] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.394660 29845 raft_consensus.cc:2243] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.395677 29845 raft_consensus.cc:2272] T 2439e7934b98446ca4e7d82381e8a499 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.398588 29845 tablet_replica.cc:333] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.400696 29845 raft_consensus.cc:2243] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.402350 29845 raft_consensus.cc:2272] T 0d63b31721b1434ea576ac0fab297089 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.405401 29845 tablet_replica.cc:333] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.406078 29845 raft_consensus.cc:2243] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.407274 29845 raft_consensus.cc:2272] T e3de96a6f20142c0a00ff9ee12bcc86c P 2c4511d66b474ee0aa41c9b1f03b66ab [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.409911 29845 tablet_replica.cc:333] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.410962 29845 raft_consensus.cc:2243] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:18.412163 29845 raft_consensus.cc:2272] T 45809554a739456ea0a596484ede5ac7 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.415104 29845 tablet_replica.cc:333] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.415864 29845 raft_consensus.cc:2243] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.416620 29845 raft_consensus.cc:2272] T 363b01e660fa4b36aa4a5112bab1ac09 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.419323 29845 tablet_replica.cc:333] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.420010 29845 raft_consensus.cc:2243] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.420902 29845 raft_consensus.cc:2272] T 205fb3f3a3844073a49da13542da2989 P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:18.425102 29845 tablet_replica.cc:333] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab: stopping tablet replica
I20260709 12:27:18.425743 29845 raft_consensus.cc:2243] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:18.426551 29845 raft_consensus.cc:2272] T 60b6187f32774daba89763099dabeb2a P 2c4511d66b474ee0aa41c9b1f03b66ab [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:18.470343 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:18.656579 31664 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:18.714210 31664 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:18.741946 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:18.807854 31664 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:18.823856 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:18.842984  6667 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:18.843566  6667 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:27:18.853971 31106 leader_election.cc:336] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:18.865168 31109 leader_election.cc:336] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:18.865687 31109 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:18.846055  6667 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:18.869715  6639 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:18.912596 31109 consensus_peers.cc:597] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.064085  6667 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.064643  6667 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.066830  6667 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.077383 31106 leader_election.cc:336] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.079289 31109 leader_election.cc:336] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.079816 31109 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.081090  6667 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:19.114760  6667 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.115242  6667 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.117408  6667 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:19.125623  6667 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.126091  6667 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.128181  6667 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.130190 31109 leader_election.cc:336] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:19.132781 31109 leader_election.cc:336] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:19.133191 31106 leader_election.cc:336] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.133688 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.134544  6667 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:19.137259  6664 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.137804  6664 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.139772  6664 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.159329 31661 leader_election.cc:336] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.169219 31106 leader_election.cc:336] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.170037 31106 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:27:19.171665 31664 leader_election.cc:336] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.172215  6667 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:19.174784 31664 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.182385  6664 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:19.214120  6664 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.214761  6664 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:19.224207 31661 leader_election.cc:336] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.225318  6664 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:19.230672 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
W20260709 12:27:19.237511 31109 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.248142 31664 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:27:19.251780  6664 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:19.255069 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.258410  6667 raft_consensus.cc:493] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.258962  6667 raft_consensus.cc:515] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.261076  6667 leader_election.cc:290] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.266527 31109 leader_election.cc:336] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:19.274076 31664 leader_election.cc:336] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.283329  6671 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.299225  6671 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.301515  6671 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:19.307343 31661 leader_election.cc:336] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.288038 31106 leader_election.cc:336] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.308442 31106 leader_election.cc:304] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:27:19.298583 31109 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:27:19.310592  6667 raft_consensus.cc:2749] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:19.320089 31142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:19.326267 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:27:19.327116  6671 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:19.328974 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.353979  6667 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.371110  6667 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.374079  6667 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:27:19.379027 31109 leader_election.cc:336] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:19.380213 31664 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:34803, user_credentials={real_user=slave}} blocked reactor thread for 51290.5us
I20260709 12:27:19.390472 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:19.391089 31696 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 27.
I20260709 12:27:19.392848  6639 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
W20260709 12:27:19.393249 31106 leader_election.cc:336] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.393309  6639 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.393778 31106 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.394428  6667 raft_consensus.cc:2749] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:19.397745  6639 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.401377 31106 leader_election.cc:336] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.401553 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:19.418802 31109 leader_election.cc:336] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.419346 31109 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.420089  6639 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:19.473235 31109 consensus_peers.cc:597] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.484014  6672 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.484541  6672 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.486725  6672 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.491474 31106 leader_election.cc:336] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.495287 31109 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.495783 31109 leader_election.cc:336] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.496228 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.496973  6672 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:19.510574 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:27:19.516633  6671 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.517194  6671 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.519215  6671 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.523442 31661 leader_election.cc:336] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.526736 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:19.529086 31664 leader_election.cc:336] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.530012 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.532994  6671 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:19.595086  6664 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.595615  6664 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.597666  6664 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.610989 31664 leader_election.cc:336] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:19.611768 31661 leader_election.cc:336] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.612217 31661 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.612960  6671 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:19.665916 29845 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 31090)
I20260709 12:27:19.668267  6639 raft_consensus.cc:493] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.668773  6639 raft_consensus.cc:515] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.670645  6639 leader_election.cc:290] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.675532 31109 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:19.675964 31106 leader_election.cc:336] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.679347 31109 leader_election.cc:336] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.679786 31109 leader_election.cc:304] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.680864  6639 raft_consensus.cc:2749] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:19.692595  6672 raft_consensus.cc:493] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.693092  6672 raft_consensus.cc:515] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.695153  6672 leader_election.cc:290] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.700191 31106 leader_election.cc:336] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.705564 31109 leader_election.cc:336] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.706820 31109 leader_election.cc:304] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.707553  6639 raft_consensus.cc:2749] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:19.709190  6672 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.709955  6672 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.713973  6672 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
W20260709 12:27:19.721808 31106 leader_election.cc:336] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.722570  6672 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.724814 31689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:19.725440 31689 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 24.
I20260709 12:27:19.723099  6672 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.731719  6672 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:19.734412 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
W20260709 12:27:19.735998 31106 leader_election.cc:336] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.740526  6664 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.741272  6664 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.745142 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:19.743964  6664 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:19.751055  6639 raft_consensus.cc:493] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.751695  6639 raft_consensus.cc:515] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.754041  6639 leader_election.cc:290] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:19.749967 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:27:19.756539  6672 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:19.756884  6672 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.757337  6672 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:19.762755 31106 leader_election.cc:336] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.766209 31109 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, a517aadb0ae04b89a13779d7898dfa0c
W20260709 12:27:19.772531 31661 leader_election.cc:336] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.772985 31661 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:27:19.774420  6667 raft_consensus.cc:2749] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:19.778544 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.779917  6673 raft_consensus.cc:2749] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:19.790416  6672 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:19.798775 31109 leader_election.cc:336] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.799229 31109 leader_election.cc:304] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.800267  6667 raft_consensus.cc:2749] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:19.804023  6672 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:19.805833 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e39858333670419fb20dc8adde042dd2" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:27:19.806411 31696 raft_consensus.cc:3060] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.807950  6673 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.808532  6673 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:19.809178 31106 leader_election.cc:336] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.810635  6673 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:19.812224 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b" is_pre_election: true
I20260709 12:27:19.812803 31140 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 24.
I20260709 12:27:19.814589 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 
I20260709 12:27:19.815857 31696 raft_consensus.cc:2468] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 25.
I20260709 12:27:19.816167  6664 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:19.816617  6664 raft_consensus.cc:493] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.816992  6664 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.823980  6674 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.824520  6674 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.826617  6674 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.830431 31661 leader_election.cc:336] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.840528 31661 leader_election.cc:336] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.841331 31664 leader_election.cc:336] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.841780 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.842597  6673 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:19.847079  6664 raft_consensus.cc:515] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.857652  6672 raft_consensus.cc:493] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.862565  6672 raft_consensus.cc:515] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:19.864646  6672 leader_election.cc:290] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:19.865239 31661 leader_election.cc:336] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.857864 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.859115 31109 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:27:19.866484  6672 raft_consensus.cc:2804] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:19.867193  6672 raft_consensus.cc:697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:27:19.868175  6672 consensus_queue.cc:237] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.869896  6664 leader_election.cc:290] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:19.870481 31140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d63b31721b1434ea576ac0fab297089" candidate_uuid: "a517aadb0ae04b89a13779d7898dfa0c" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ce27a72886964e4287059a1d66bd5b6b"
I20260709 12:27:19.871014 31140 raft_consensus.cc:3060] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:19.871651 31106 leader_election.cc:336] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:19.874094 31109 consensus_peers.cc:597] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:19.879508 31140 raft_consensus.cc:2468] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a517aadb0ae04b89a13779d7898dfa0c in term 25.
I20260709 12:27:19.880988 31664 leader_election.cc:304] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:27:19.881106  6639 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.881788  6673 raft_consensus.cc:2804] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:19.881629  6639 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.882328  6673 raft_consensus.cc:697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Becoming Leader. State: Replica: a517aadb0ae04b89a13779d7898dfa0c, State: Running, Role: LEADER
I20260709 12:27:19.883208  6673 consensus_queue.cc:237] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:19.883688 31109 leader_election.cc:336] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:19.883801  6639 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:19.884158 31109 leader_election.cc:304] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:19.887429 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:19.888015 31696 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 18.
W20260709 12:27:19.890340 31106 leader_election.cc:336] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.886648  6639 raft_consensus.cc:2749] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:19.898211 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:27:19.907083  6672 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:19.907500  6672 raft_consensus.cc:493] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:19.907840  6672 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:19.908205 30884 catalog_manager.cc:5697] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c reported cstate change: term changed from 24 to 25, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68). New cstate: current_term: 25 leader_uuid: "a517aadb0ae04b89a13779d7898dfa0c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:19.915004  6672 raft_consensus.cc:515] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.915251 30880 catalog_manager.cc:5697] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 24 to 25, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 25 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:19.924361 31106 leader_election.cc:336] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:19.926685  6672 leader_election.cc:290] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Requested vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:19.931337 31697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45809554a739456ea0a596484ede5ac7" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c"
I20260709 12:27:19.931910 31697 raft_consensus.cc:3060] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:19.939589 31697 raft_consensus.cc:2468] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 19.
I20260709 12:27:19.942075 31109 leader_election.cc:304] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab
I20260709 12:27:19.943154  6672 raft_consensus.cc:2804] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:19.943976  6672 raft_consensus.cc:697] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Becoming Leader. State: Replica: ce27a72886964e4287059a1d66bd5b6b, State: Running, Role: LEADER
I20260709 12:27:19.948405  6672 consensus_queue.cc:237] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:19.992743 30880 catalog_manager.cc:5697] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b reported cstate change: term changed from 18 to 19, leader changed from 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66) to ce27a72886964e4287059a1d66bd5b6b (127.29.37.67). New cstate: current_term: 19 leader_uuid: "ce27a72886964e4287059a1d66bd5b6b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:20.057581 31106 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.084169 31664 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.145738 31109 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.155901 31106 consensus_peers.cc:597] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.164863 31109 consensus_peers.cc:597] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.178404 31661 consensus_peers.cc:597] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.180977 31106 consensus_peers.cc:597] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.227273 31661 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.231326 31106 consensus_peers.cc:597] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.236914 31109 consensus_peers.cc:597] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:20.237483 31106 consensus_peers.cc:597] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.251600 31661 consensus_peers.cc:597] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.253024 31109 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:20.267468 31106 consensus_peers.cc:597] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.271970 31109 consensus_peers.cc:597] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.286155 31661 consensus_peers.cc:597] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.313879 31109 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.329197 31661 consensus_peers.cc:597] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:20.334817 31140 raft_consensus.cc:1275] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Refusing update from remote peer a517aadb0ae04b89a13779d7898dfa0c: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:20.342061  6674 consensus_queue.cc:1048] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:27:20.341722 31109 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:20.360486 31661 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.368366 31661 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.372041 31106 consensus_peers.cc:597] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.383101 31661 consensus_peers.cc:597] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.383613 31106 consensus_peers.cc:597] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.401592 31661 consensus_peers.cc:597] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.411041 31661 consensus_peers.cc:597] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:20.430795 31661 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:20.453299 31696 raft_consensus.cc:1275] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 12:27:20.456889 31661 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.458642 31106 consensus_peers.cc:597] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:20.477952  6639 consensus_queue.cc:1048] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:27:20.487620 31106 consensus_peers.cc:597] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.494212 31106 consensus_peers.cc:597] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.497285 31109 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:20.496035 31106 consensus_peers.cc:597] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:20.506107 31661 consensus_peers.cc:597] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.516984 31106 consensus_peers.cc:597] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.522768 31106 consensus_peers.cc:597] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:20.546345 31697 raft_consensus.cc:1275] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Refusing update from remote peer ce27a72886964e4287059a1d66bd5b6b: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 19 index: 18. (index mismatch)
I20260709 12:27:20.548578  6639 consensus_queue.cc:1048] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:27:20.652756 31661 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.680858 31661 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:20.689483 31106 consensus_peers.cc:597] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:20.778532  6688 raft_consensus.cc:493] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:20.779093  6688 raft_consensus.cc:515] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:20.781286  6688 leader_election.cc:290] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:20.788494 31106 leader_election.cc:336] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:20.790956 29845 tablet_server.cc:196] TabletServer@127.29.37.66:0 shutdown complete.
W20260709 12:27:20.819582 31109 leader_election.cc:336] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:20.820268 31109 leader_election.cc:304] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:27:20.822031 31106 consensus_peers.cc:597] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:20.822654  6672 raft_consensus.cc:2749] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:20.825448  6683 raft_consensus.cc:493] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:20.825973  6683 raft_consensus.cc:515] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:20.828044  6683 leader_election.cc:290] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:20.833603 31664 leader_election.cc:336] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:20.882963 31661 leader_election.cc:336] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:20.883524 31661 leader_election.cc:304] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:20.884322  6683 raft_consensus.cc:2749] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:20.956149  6672 raft_consensus.cc:493] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:20.956704  6672 raft_consensus.cc:515] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:20.965008  6672 leader_election.cc:290] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:20.965672 31109 leader_election.cc:336] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:20.974134  6672 raft_consensus.cc:493] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:20.974736  6672 raft_consensus.cc:515] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
W20260709 12:27:20.981979 31661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111) [suppressed 603 similar messages]
I20260709 12:27:20.983646  6672 leader_election.cc:290] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:20.990231 31109 leader_election.cc:336] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:20.992734 31106 leader_election.cc:336] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:20.995189 31106 leader_election.cc:304] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:20.995992  6672 raft_consensus.cc:2749] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:21.009560 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:21.019232 31106 leader_election.cc:336] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:21.019848 31106 leader_election.cc:304] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:21.020259 29845 tablet_server.cc:179] TabletServer@127.29.37.67:0 shutting down...
I20260709 12:27:21.020661  6672 raft_consensus.cc:2749] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:21.028360 31664 consensus_peers.cc:597] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:21.035468  6672 raft_consensus.cc:493] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:21.036058  6672 raft_consensus.cc:515] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:21.043464  6672 leader_election.cc:290] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:21.049397 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:21.049939 31109 leader_election.cc:336] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:21.052098 31106 leader_election.cc:336] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:21.052537 31106 leader_election.cc:304] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:21.055235  6688 raft_consensus.cc:2749] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:21.057770 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:21.065697  6688 raft_consensus.cc:493] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:21.066382  6688 raft_consensus.cc:515] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:21.068437  6688 leader_election.cc:290] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:21.072666 31109 leader_election.cc:336] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:21.073590 31106 leader_election.cc:336] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:21.074007 31106 leader_election.cc:304] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:21.074815  6688 raft_consensus.cc:2749] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:21.084836 31664 consensus_peers.cc:597] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:21.085505 31664 consensus_peers.cc:597] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:21.108753 31664 consensus_peers.cc:597] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:21.122378 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:21.123553 29845 tablet_replica.cc:333] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.124559 29845 raft_consensus.cc:2243] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.125432 29845 raft_consensus.cc:2272] T a2c77b15766d44f585ab3cb57444a324 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.128160 29845 tablet_replica.cc:333] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.128957 29845 raft_consensus.cc:2243] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.129993 29845 raft_consensus.cc:2272] T 0ec17bf7d52c4cb69f4d488858fecc8b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.132474 29845 tablet_replica.cc:333] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.133191 29845 raft_consensus.cc:2243] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.133826 29845 raft_consensus.cc:2272] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.136422 29845 tablet_replica.cc:333] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.137198 29845 raft_consensus.cc:2243] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.138276 29845 raft_consensus.cc:2272] T d521786b7ccd41da81f7fd0d175bf582 P ce27a72886964e4287059a1d66bd5b6b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.140725 29845 tablet_replica.cc:333] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.141458 29845 raft_consensus.cc:2243] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.142014 29845 raft_consensus.cc:2272] T 76399531471a4790a56d52a6ea052864 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.144510 29845 tablet_replica.cc:333] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.145248 29845 raft_consensus.cc:2243] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.145824 29845 raft_consensus.cc:2272] T d50d1465e4b84136bf90de700a93ef2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.149089 29845 tablet_replica.cc:333] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.149832 29845 raft_consensus.cc:2243] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.150835 29845 raft_consensus.cc:2272] T 95980265645c4f7a8d6b201f0ae58174 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.153218 29845 tablet_replica.cc:333] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.153940 29845 raft_consensus.cc:2243] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.154712 29845 raft_consensus.cc:2272] T 1aff3791e3404a7e92be14f04f9674ec P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:21.156723 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:21.157223 29845 tablet_replica.cc:333] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
W20260709 12:27:21.158190 31664 consensus_peers.cc:597] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:21.158578 29845 raft_consensus.cc:2243] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:21.159330 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:21.159405 29845 raft_consensus.cc:2272] T 635264f51b2d4410a1da9b76606a6350 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:21.161149 31664 consensus_peers.cc:597] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:21.162261 29845 tablet_replica.cc:333] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.163827 29845 raft_consensus.cc:2243] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.165198  6688 raft_consensus.cc:493] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:21.165786  6688 raft_consensus.cc:515] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } }
I20260709 12:27:21.167805  6688 leader_election.cc:290] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
I20260709 12:27:21.165249 29845 raft_consensus.cc:2272] T 4950edf9dfb9412981a0b325bf866deb P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.170557 29845 tablet_replica.cc:333] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.171347 29845 raft_consensus.cc:2243] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.172312 29845 raft_consensus.cc:2272] T 044e61a321dd4521be7f9ee796ca44dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:21.173367 31106 leader_election.cc:336] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:21.175855 29845 tablet_replica.cc:333] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.176693 29845 raft_consensus.cc:2243] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.177219 29845 raft_consensus.cc:2272] T f311574d640f4ff69dd8e9525a8fe8c1 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.179864 29845 tablet_replica.cc:333] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.180626 29845 raft_consensus.cc:2243] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.181684 29845 raft_consensus.cc:2272] T 676d422dffb24e3cb16409a80bb28758 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.184158 29845 tablet_replica.cc:333] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.184935 29845 raft_consensus.cc:2243] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:21.185377 31109 leader_election.cc:336] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:21.187602 31109 leader_election.cc:304] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:21.188406  6688 raft_consensus.cc:2749] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:21.185585 29845 raft_consensus.cc:2272] T 9dab2cd673924dc2a2f8622e76701947 P ce27a72886964e4287059a1d66bd5b6b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.191417 29845 tablet_replica.cc:333] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.192260 29845 raft_consensus.cc:2243] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.193339 29845 raft_consensus.cc:2272] T 4a3e0211fca8429086a4403f1ec42a59 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.196619 29845 tablet_replica.cc:333] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.197448 29845 raft_consensus.cc:2243] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.198145 29845 raft_consensus.cc:2272] T 9bb92178abcc4ef385f771bd04448a5b P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.200747 29845 tablet_replica.cc:333] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.201608 29845 raft_consensus.cc:2243] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.202790 29845 raft_consensus.cc:2272] T d8e9bb54e628458a94d623c051d36c1a P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.205297 29845 tablet_replica.cc:333] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.206111 29845 raft_consensus.cc:2243] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.206728 29845 raft_consensus.cc:2272] T c6dbea78f5c04486a0fc3f17e2260e2c P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.209178 29845 tablet_replica.cc:333] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.209954 29845 raft_consensus.cc:2243] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.211063 29845 raft_consensus.cc:2272] T 563f5eec53c14273959c9883d2423cc9 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.213680 29845 tablet_replica.cc:333] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.214540 29845 raft_consensus.cc:2243] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.215379 29845 raft_consensus.cc:2272] T 0c6a2f53447447459d88ab978179c65c P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.218645 29845 tablet_replica.cc:333] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
W20260709 12:27:21.219416 31664 consensus_peers.cc:597] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:21.219465 29845 raft_consensus.cc:2243] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.221467 29845 raft_consensus.cc:2272] T d48e42d074cd496f996172cff1a0c835 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.223981 29845 tablet_replica.cc:333] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.224773 29845 raft_consensus.cc:2243] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.225719 29845 raft_consensus.cc:2272] T ff2c202e27d24d1baad8ddf5d9db4391 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.228418 29845 tablet_replica.cc:333] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.229285 29845 raft_consensus.cc:2243] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.229936 29845 raft_consensus.cc:2272] T 330f8038ee4e498ebefd9d773f4e7199 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.232471 29845 tablet_replica.cc:333] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.233206 29845 raft_consensus.cc:2243] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.233827 29845 raft_consensus.cc:2272] T 39b66c047b4e42afad80738552eee0dc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.236299 29845 tablet_replica.cc:333] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.237037 29845 raft_consensus.cc:2243] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.238123 29845 raft_consensus.cc:2272] T 5bd1ada4571045f6ba305e207cccddcf P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.239578  6672 raft_consensus.cc:493] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:21.240119  6672 raft_consensus.cc:515] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } attrs { replace: true } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:21.241001 29845 tablet_replica.cc:333] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.241855 29845 raft_consensus.cc:2243] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.242704  6672 leader_election.cc:290] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), a517aadb0ae04b89a13779d7898dfa0c (127.29.37.68:35417)
I20260709 12:27:21.242933 29845 raft_consensus.cc:2272] T f93e9d74965c45a19eb84bff3d1f9825 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.245671 29845 tablet_replica.cc:333] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.246519 29845 raft_consensus.cc:2243] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.247148 29845 raft_consensus.cc:2272] T 82382d1e93d244cd8307164a5ac891b0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:21.247694 31106 leader_election.cc:336] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:21.249707 29845 tablet_replica.cc:333] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.250571 29845 raft_consensus.cc:2243] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.252336 29845 raft_consensus.cc:2272] T e39858333670419fb20dc8adde042dd2 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.254417 31696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e98467ef2b2b4bab954b704025d182b4" candidate_uuid: "ce27a72886964e4287059a1d66bd5b6b" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a517aadb0ae04b89a13779d7898dfa0c" is_pre_election: true
I20260709 12:27:21.255018 31696 raft_consensus.cc:2468] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce27a72886964e4287059a1d66bd5b6b in term 27.
W20260709 12:27:21.256873 31664 consensus_peers.cc:597] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:21.259716 29845 tablet_replica.cc:333] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.260535 29845 raft_consensus.cc:2243] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.261158 29845 raft_consensus.cc:2272] T bc9ac1b8eaf94e99a3eee16d0ca65113 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.263667 29845 tablet_replica.cc:333] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.264371 29845 raft_consensus.cc:2243] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.265247 29845 raft_consensus.cc:2272] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:21.267625 31109 leader_election.cc:336] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:21.267660 29845 tablet_replica.cc:333] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.268163 31109 leader_election.cc:304] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c, ce27a72886964e4287059a1d66bd5b6b; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:21.268838 29845 raft_consensus.cc:2243] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:27:21.269178 31664 consensus_peers.cc:597] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:21.270155 29845 raft_consensus.cc:2272] T a7f0812b392640ecab5f5257bb0e41ee P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:21.271443 31664 consensus_peers.cc:597] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:21.268867 31109 raft_consensus.cc:2686] T e98467ef2b2b4bab954b704025d182b4 P ce27a72886964e4287059a1d66bd5b6b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:21.272622 29845 tablet_replica.cc:333] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.273473 29845 raft_consensus.cc:2243] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.274919 29845 raft_consensus.cc:2272] T dd024d82eb9c473ba31252aeeb287fc0 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.277004 29845 tablet_replica.cc:333] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.277549 29845 raft_consensus.cc:2243] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.278105 29845 raft_consensus.cc:2272] T 4d522233ee4c4536b4fc944d7866c4d9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.279897 29845 tablet_replica.cc:333] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.280645 29845 raft_consensus.cc:2243] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.281306 29845 raft_consensus.cc:2272] T 4ff17d7bb34443adb04e437ed0079e51 P ce27a72886964e4287059a1d66bd5b6b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.283919 29845 tablet_replica.cc:333] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.284636 29845 raft_consensus.cc:2243] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.285660 29845 raft_consensus.cc:2272] T 48047b2b139146c38a821328cf7bd148 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.288192 29845 tablet_replica.cc:333] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.288910 29845 raft_consensus.cc:2243] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.289420 29845 raft_consensus.cc:2272] T 0d63b31721b1434ea576ac0fab297089 P ce27a72886964e4287059a1d66bd5b6b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.291610 29845 tablet_replica.cc:333] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.292111 29845 raft_consensus.cc:2243] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.293047 29845 raft_consensus.cc:2272] T 72d9d9cc8be6411883e9f3f17ef865eb P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.295087 29845 tablet_replica.cc:333] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.295732 29845 raft_consensus.cc:2243] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.296841 29845 raft_consensus.cc:2272] T 41116db50e8c4b9598abe43f5d36bf12 P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.299418 29845 tablet_replica.cc:333] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.300149 29845 raft_consensus.cc:2243] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.301136 29845 raft_consensus.cc:2272] T 45809554a739456ea0a596484ede5ac7 P ce27a72886964e4287059a1d66bd5b6b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.304207 29845 tablet_replica.cc:333] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.304767 29845 raft_consensus.cc:2243] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.305598 29845 raft_consensus.cc:2272] T ba8a01bf2a514da2805eb75a7c28c412 P ce27a72886964e4287059a1d66bd5b6b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.307569 29845 tablet_replica.cc:333] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.308058 29845 raft_consensus.cc:2243] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.308511 29845 raft_consensus.cc:2272] T b8ba55d3e2574f7a890f7a956ba4e74f P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.310441 29845 tablet_replica.cc:333] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.310905 29845 raft_consensus.cc:2243] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.311414 29845 raft_consensus.cc:2272] T b4162f35b54e4acc8abf0a1f9fc084a9 P ce27a72886964e4287059a1d66bd5b6b [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.313270 29845 tablet_replica.cc:333] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.313742 29845 raft_consensus.cc:2243] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.314253 29845 raft_consensus.cc:2272] T 7ea7681ba29145eba9db5bc137cf18b4 P ce27a72886964e4287059a1d66bd5b6b [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.316675 29845 tablet_replica.cc:333] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.317831 29845 raft_consensus.cc:2243] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.318890 29845 raft_consensus.cc:2272] T 05394269d1a4461bb79d6f93ab25ff4e P ce27a72886964e4287059a1d66bd5b6b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.320982 29845 tablet_replica.cc:333] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.321475 29845 raft_consensus.cc:2243] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.321919 29845 raft_consensus.cc:2272] T 924fd2989ab145f09d211890a85a03fc P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.323895 29845 tablet_replica.cc:333] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.324366 29845 raft_consensus.cc:2243] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.326424 29845 raft_consensus.cc:2272] T 7d43e79f1b224452ad15ff525d3a0bd9 P ce27a72886964e4287059a1d66bd5b6b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.328787 29845 tablet_replica.cc:333] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b: stopping tablet replica
I20260709 12:27:21.329893 29845 raft_consensus.cc:2243] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:21.330673 29845 raft_consensus.cc:2272] T 76fe9dfd8b4c4785a354982ad7e1faa4 P ce27a72886964e4287059a1d66bd5b6b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:21.331069 31664 consensus_peers.cc:597] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:21.339074 31664 consensus_peers.cc:597] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:21.388875 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:21.397635  6683 raft_consensus.cc:493] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:21.398363  6683 raft_consensus.cc:515] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:21.400588  6683 leader_election.cc:290] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:21.406926 31661 leader_election.cc:336] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:21.409858 31664 leader_election.cc:336] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:21.410385 31664 leader_election.cc:304] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:21.411156  6683 raft_consensus.cc:2749] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:21.426162 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:21.462404 31664 consensus_peers.cc:597] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:21.470253 31664 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:21.499776 31664 consensus_peers.cc:597] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c -> Peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Couldn't send request to peer ce27a72886964e4287059a1d66bd5b6b. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:21.681475 31664 consensus_peers.cc:597] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:21.731009 31664 consensus_peers.cc:597] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:21.795396  6674 raft_consensus.cc:493] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:21.795969  6674 raft_consensus.cc:515] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:21.798018  6674 leader_election.cc:290] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:21.813483 31661 leader_election.cc:336] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:21.826336 31664 leader_election.cc:336] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:21.826898 31664 leader_election.cc:304] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:21.827672  6674 raft_consensus.cc:2749] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:21.945813 31664 consensus_peers.cc:597] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:22.013111  6683 raft_consensus.cc:493] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2c4511d66b474ee0aa41c9b1f03b66ab)
I20260709 12:27:22.013666  6683 raft_consensus.cc:515] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.015784  6683 leader_election.cc:290] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157)
W20260709 12:27:22.021333 31661 leader_election.cc:336] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:22.029405 31664 leader_election.cc:336] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:22.029912 31664 leader_election.cc:304] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:22.030654  6683 raft_consensus.cc:2749] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:22.120744 31664 consensus_peers.cc:597] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:27:22.264611  6683 raft_consensus.cc:493] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.265192  6683 raft_consensus.cc:515] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.268456  6683 leader_election.cc:290] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:22.274550  6674 raft_consensus.cc:493] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.275890  6674 raft_consensus.cc:515] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.277958  6674 leader_election.cc:290] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:22.275339 31661 leader_election.cc:336] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:22.292088  6683 raft_consensus.cc:493] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.292646  6683 raft_consensus.cc:515] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.294760  6683 leader_election.cc:290] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:22.320642 31664 leader_election.cc:336] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:22.321192 31664 leader_election.cc:304] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:22.321923  6683 raft_consensus.cc:2749] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:22.326370  6674 raft_consensus.cc:493] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.327283  6674 raft_consensus.cc:515] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.329280  6674 leader_election.cc:290] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:22.336244 31661 leader_election.cc:336] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:22.338886 31664 leader_election.cc:336] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:22.340732 31664 leader_election.cc:336] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
W20260709 12:27:22.341723 31664 leader_election.cc:336] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
I20260709 12:27:22.342156 31664 leader_election.cc:304] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:22.345126  6683 raft_consensus.cc:2749] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:22.353516 31664 leader_election.cc:336] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:22.362394 31664 leader_election.cc:336] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
I20260709 12:27:22.362834 31664 leader_election.cc:304] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:22.363543  6683 raft_consensus.cc:2749] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:22.371277 31664 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.371778 31664 consensus_peers.cc:597] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c -> Peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Couldn't send request to peer e9cccf7eb2a545ef8c2169b67ff261bf. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:22.378454 31664 leader_election.cc:336] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
I20260709 12:27:22.378866 31664 leader_election.cc:304] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:27:22.379592  6683 raft_consensus.cc:2749] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:22.445402 29845 tablet_server.cc:196] TabletServer@127.29.37.67:0 shutdown complete.
I20260709 12:27:22.486624  6683 raft_consensus.cc:493] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.487461  6683 raft_consensus.cc:515] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.498358  6674 raft_consensus.cc:493] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.498869  6674 raft_consensus.cc:515] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.501291  6674 leader_election.cc:290] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:22.502382 31661 leader_election.cc:336] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:22.503508  6683 leader_election.cc:290] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:22.506126  6683 raft_consensus.cc:493] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.507669  6683 raft_consensus.cc:515] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.516885  6683 leader_election.cc:290] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:22.517405 31661 leader_election.cc:336] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:22.530393  6683 raft_consensus.cc:493] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.531040  6683 raft_consensus.cc:515] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.533466  6683 leader_election.cc:290] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:22.540486 31661 leader_election.cc:336] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
W20260709 12:27:22.543663 31664 leader_election.cc:336] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
W20260709 12:27:22.547024 31664 leader_election.cc:336] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
I20260709 12:27:22.547461 31664 leader_election.cc:304] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:27:22.548240  6683 raft_consensus.cc:2749] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:22.553756 31664 leader_election.cc:336] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
I20260709 12:27:22.554265 31664 leader_election.cc:304] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:27:22.555058  6683 raft_consensus.cc:2749] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:22.559429 31664 leader_election.cc:336] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:22.559964 31664 leader_election.cc:304] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:22.560752  6683 raft_consensus.cc:2749] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:22.563606  6683 raft_consensus.cc:493] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.564128  6683 raft_consensus.cc:515] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "2c4511d66b474ee0aa41c9b1f03b66ab" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45157 } attrs { replace: true } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
W20260709 12:27:22.573378 31661 leader_election.cc:336] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111)
I20260709 12:27:22.574693  6683 leader_election.cc:290] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
I20260709 12:27:22.575642 29845 tablet_server.cc:179] TabletServer@127.29.37.68:0 shutting down...
W20260709 12:27:22.579260 31664 leader_election.cc:336] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
I20260709 12:27:22.579738 31664 leader_election.cc:304] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, ce27a72886964e4287059a1d66bd5b6b
I20260709 12:27:22.580475  6683 raft_consensus.cc:2749] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:22.583225  6683 raft_consensus.cc:493] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ce27a72886964e4287059a1d66bd5b6b)
I20260709 12:27:22.583702  6683 raft_consensus.cc:515] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9cccf7eb2a545ef8c2169b67ff261bf" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 34803 } } peers { permanent_uuid: "ce27a72886964e4287059a1d66bd5b6b" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 36565 } } peers { permanent_uuid: "a517aadb0ae04b89a13779d7898dfa0c" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 35417 } attrs { promote: false } }
I20260709 12:27:22.585906  6683 leader_election.cc:290] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803), ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565)
W20260709 12:27:22.596781 31664 leader_election.cc:336] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:22.597673 31664 leader_election.cc:304] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: 2c4511d66b474ee0aa41c9b1f03b66ab, e9cccf7eb2a545ef8c2169b67ff261bf
W20260709 12:27:22.602572 31664 leader_election.cc:336] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
I20260709 12:27:22.603296  6683 raft_consensus.cc:2749] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:22.607394 31664 leader_election.cc:336] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
W20260709 12:27:22.608333 31664 leader_election.cc:336] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer ce27a72886964e4287059a1d66bd5b6b (127.29.37.67:36565): Network error: Client connection negotiation failed: client connection to 127.29.37.67:36565: connect: Connection refused (error 111)
I20260709 12:27:22.608714 31664 leader_election.cc:304] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a517aadb0ae04b89a13779d7898dfa0c; no voters: ce27a72886964e4287059a1d66bd5b6b, e9cccf7eb2a545ef8c2169b67ff261bf
I20260709 12:27:22.609500  6674 raft_consensus.cc:2749] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:22.615139 31661 consensus_peers.cc:597] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:22.636560 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:22.637439 29845 tablet_replica.cc:333] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.638437 29845 raft_consensus.cc:2243] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.639269 29845 raft_consensus.cc:2272] T 676d422dffb24e3cb16409a80bb28758 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.642375 29845 tablet_replica.cc:333] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.643261 29845 raft_consensus.cc:2243] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.643863 29845 raft_consensus.cc:2272] T e39858333670419fb20dc8adde042dd2 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.646272 29845 tablet_replica.cc:333] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.647094 29845 raft_consensus.cc:2243] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.648160 29845 raft_consensus.cc:2272] T a2c77b15766d44f585ab3cb57444a324 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.651302 29845 tablet_replica.cc:333] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.652107 29845 raft_consensus.cc:2243] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.653172 29845 raft_consensus.cc:2272] T 0d63b31721b1434ea576ac0fab297089 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.655696 29845 tablet_replica.cc:333] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.656489 29845 raft_consensus.cc:2243] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.657446 29845 raft_consensus.cc:2272] T 13959daae3c149d6ae8248d1f3ca6e59 P a517aadb0ae04b89a13779d7898dfa0c [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.661203 29845 tablet_replica.cc:333] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.661993 29845 raft_consensus.cc:2243] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.662652 29845 raft_consensus.cc:2272] T 05394269d1a4461bb79d6f93ab25ff4e P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:22.665380 31661 consensus_peers.cc:597] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:22.666853 29845 tablet_replica.cc:333] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.667678 29845 raft_consensus.cc:2243] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.668800 29845 raft_consensus.cc:2272] T 4f57ce098bf541c4b1ca32a9c4b8cec0 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.671255 29845 tablet_replica.cc:333] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.671980 29845 raft_consensus.cc:2243] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.672919 29845 raft_consensus.cc:2272] T 363b01e660fa4b36aa4a5112bab1ac09 P a517aadb0ae04b89a13779d7898dfa0c [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.675375 29845 tablet_replica.cc:333] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.676137 29845 raft_consensus.cc:2243] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.677114 29845 raft_consensus.cc:2272] T 044e61a321dd4521be7f9ee796ca44dc P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.679538 29845 tablet_replica.cc:333] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.680387 29845 raft_consensus.cc:2243] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.681015 29845 raft_consensus.cc:2272] T 0ec17bf7d52c4cb69f4d488858fecc8b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.683655 29845 tablet_replica.cc:333] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.684259 29845 raft_consensus.cc:2243] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.684792 29845 raft_consensus.cc:2272] T dd024d82eb9c473ba31252aeeb287fc0 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.687458 29845 tablet_replica.cc:333] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.688391 29845 raft_consensus.cc:2243] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.689627 29845 raft_consensus.cc:2272] T 635264f51b2d4410a1da9b76606a6350 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.692552 29845 tablet_replica.cc:333] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.693120 29845 raft_consensus.cc:2243] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.694017 29845 raft_consensus.cc:2272] T 205fb3f3a3844073a49da13542da2989 P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.696081 29845 tablet_replica.cc:333] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.696645 29845 raft_consensus.cc:2243] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.697165 29845 raft_consensus.cc:2272] T e3de96a6f20142c0a00ff9ee12bcc86c P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.699134 29845 tablet_replica.cc:333] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.699826 29845 raft_consensus.cc:2243] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.700333 29845 raft_consensus.cc:2272] T ba8a01bf2a514da2805eb75a7c28c412 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.702814 29845 tablet_replica.cc:333] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.703557 29845 raft_consensus.cc:2243] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.704106 29845 raft_consensus.cc:2272] T 45809554a739456ea0a596484ede5ac7 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.706575 29845 tablet_replica.cc:333] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
W20260709 12:27:22.706571 31661 consensus_peers.cc:597] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:22.707319 29845 raft_consensus.cc:2243] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.707971 29845 raft_consensus.cc:2272] T e98467ef2b2b4bab954b704025d182b4 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.710669 29845 tablet_replica.cc:333] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.711488 29845 raft_consensus.cc:2243] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 27 LEADER]: Raft consensus shutting down.
W20260709 12:27:22.717072 31661 consensus_peers.cc:597] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:22.739603 29845 raft_consensus.cc:2272] T 544dffe603ca4dce9d6697ea6b15c925 P a517aadb0ae04b89a13779d7898dfa0c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.742659 29845 tablet_replica.cc:333] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.743462 29845 raft_consensus.cc:2243] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.744127 29845 raft_consensus.cc:2272] T 41116db50e8c4b9598abe43f5d36bf12 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.746934 29845 tablet_replica.cc:333] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.747689 29845 raft_consensus.cc:2243] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.748243 29845 raft_consensus.cc:2272] T 72d9d9cc8be6411883e9f3f17ef865eb P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.750737 29845 tablet_replica.cc:333] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.751488 29845 raft_consensus.cc:2243] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.752118 29845 raft_consensus.cc:2272] T d48e42d074cd496f996172cff1a0c835 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.754846 29845 tablet_replica.cc:333] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.755505 29845 raft_consensus.cc:2243] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.756588 29845 raft_consensus.cc:2272] T 330f8038ee4e498ebefd9d773f4e7199 P a517aadb0ae04b89a13779d7898dfa0c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.759183 29845 tablet_replica.cc:333] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.759899 29845 raft_consensus.cc:2243] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.760862 29845 raft_consensus.cc:2272] T 60b6187f32774daba89763099dabeb2a P a517aadb0ae04b89a13779d7898dfa0c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.763379 29845 tablet_replica.cc:333] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.764365 29845 raft_consensus.cc:2243] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.765045 29845 raft_consensus.cc:2272] T 4950edf9dfb9412981a0b325bf866deb P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.767663 29845 tablet_replica.cc:333] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.768263 29845 raft_consensus.cc:2243] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.769104 29845 raft_consensus.cc:2272] T 4ff17d7bb34443adb04e437ed0079e51 P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.771688 29845 tablet_replica.cc:333] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
W20260709 12:27:22.772195 31661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:45157, user_credentials={real_user=slave}} blocked reactor thread for 55170.9us
I20260709 12:27:22.772423 29845 raft_consensus.cc:2243] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.773516 29845 raft_consensus.cc:2272] T d50d1465e4b84136bf90de700a93ef2c P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.776270 29845 tablet_replica.cc:333] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.777048 29845 raft_consensus.cc:2243] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.777587 29845 raft_consensus.cc:2272] T 0afdb545bb8f485993259007a00c1841 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.780254 29845 tablet_replica.cc:333] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.781076 29845 raft_consensus.cc:2243] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.782368 29845 raft_consensus.cc:2272] T 82399103216c4d5e8d2acdbbf3cee84d P a517aadb0ae04b89a13779d7898dfa0c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.784366 29845 tablet_replica.cc:333] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.784909 29845 raft_consensus.cc:2243] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.785713 29845 raft_consensus.cc:2272] T 2439e7934b98446ca4e7d82381e8a499 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.787825 29845 tablet_replica.cc:333] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.788311 29845 raft_consensus.cc:2243] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.789142 29845 raft_consensus.cc:2272] T 76fe9dfd8b4c4785a354982ad7e1faa4 P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.791096 29845 tablet_replica.cc:333] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.791585 29845 raft_consensus.cc:2243] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.792096 29845 raft_consensus.cc:2272] T 48047b2b139146c38a821328cf7bd148 P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.794001 29845 tablet_replica.cc:333] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.794533 29845 raft_consensus.cc:2243] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.795517 29845 raft_consensus.cc:2272] T 9bb92178abcc4ef385f771bd04448a5b P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.797915 29845 tablet_replica.cc:333] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.798643 29845 raft_consensus.cc:2243] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 22 LEADER]: Raft consensus shutting down.
W20260709 12:27:22.798955 31661 consensus_peers.cc:597] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c -> Peer 2c4511d66b474ee0aa41c9b1f03b66ab (127.29.37.66:45157): Couldn't send request to peer 2c4511d66b474ee0aa41c9b1f03b66ab. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45157: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:22.799533 29845 raft_consensus.cc:2272] T 4d522233ee4c4536b4fc944d7866c4d9 P a517aadb0ae04b89a13779d7898dfa0c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.801438 29845 tablet_replica.cc:333] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.801920 29845 raft_consensus.cc:2243] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.802776 29845 raft_consensus.cc:2272] T 83f49f3c3cc44994877e69c3ff83090e P a517aadb0ae04b89a13779d7898dfa0c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.804623 29845 tablet_replica.cc:333] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.805104 29845 raft_consensus.cc:2243] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.805541 29845 raft_consensus.cc:2272] T b1c1f8d5241540fdb4fb4daa1691260f P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.807467 29845 tablet_replica.cc:333] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.807942 29845 raft_consensus.cc:2243] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.808703 29845 raft_consensus.cc:2272] T b8ba55d3e2574f7a890f7a956ba4e74f P a517aadb0ae04b89a13779d7898dfa0c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.810508 29845 tablet_replica.cc:333] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.811007 29845 raft_consensus.cc:2243] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.811442 29845 raft_consensus.cc:2272] T 8e81e4942d094afda44355ba1d9c7735 P a517aadb0ae04b89a13779d7898dfa0c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.813305 29845 tablet_replica.cc:333] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.813786 29845 raft_consensus.cc:2243] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.814684 29845 raft_consensus.cc:2272] T ff2c202e27d24d1baad8ddf5d9db4391 P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.816510 29845 tablet_replica.cc:333] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.817049 29845 raft_consensus.cc:2243] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:22.817554 29845 raft_consensus.cc:2272] T 563f5eec53c14273959c9883d2423cc9 P a517aadb0ae04b89a13779d7898dfa0c [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:22.819346 29845 tablet_replica.cc:333] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c: stopping tablet replica
I20260709 12:27:22.819804 29845 raft_consensus.cc:2243] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:22.820552 29845 raft_consensus.cc:2272] T 0c6a2f53447447459d88ab978179c65c P a517aadb0ae04b89a13779d7898dfa0c [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:23.781931 31719 debug-util.cc:398] Leaking SignalData structure 0x7b0800192060 after lost signal to thread 29848
W20260709 12:27:23.782936 31719 debug-util.cc:398] Leaking SignalData structure 0x7b08002190e0 after lost signal to thread 30924
W20260709 12:27:23.783493 31719 debug-util.cc:398] Leaking SignalData structure 0x7b0800292d80 after lost signal to thread 31722
W20260709 12:27:23.874702 29845 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 31719)
W20260709 12:27:23.876534 30876 catalog_manager.cc:4755] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676d422dffb24e3cb16409a80bb28758: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:24.654917 29845 tablet_server.cc:196] TabletServer@127.29.37.68:0 shutdown complete.
I20260709 12:27:24.751847 29845 master.cc:562] Master@127.29.37.126:44053 shutting down...
I20260709 12:27:24.772298 30878 maintenance_manager.cc:419] P 0423d91b75f1428faccc573fcbf419a1: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033385
I20260709 12:27:24.815848 30877 maintenance_manager.cc:643] P 0423d91b75f1428faccc573fcbf419a1: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.043s	user 0.027s	sys 0.013s Metrics: {"bytes_written":7023,"delete_count":0,"lbm_write_time_us":1094,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":3428,"threads_started":1,"update_count":282,"wal-append.queue_time_us":4066}
W20260709 12:27:24.817796 30938 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:27:24.818156 30938 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:27:24.818439 30938 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:27:40.275499 30876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111) [suppressed 205 similar messages]
W20260709 12:27:40.278995 30876 catalog_manager.cc:4755] TS e9cccf7eb2a545ef8c2169b67ff261bf (127.29.37.65:34803): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 676d422dffb24e3cb16409a80bb28758: Network error: Client connection negotiation failed: client connection to 127.29.37.65:34803: connect: Connection refused (error 111)
I20260709 12:27:41.832209 29845 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:41.833227 29845 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:41.833592 29845 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0423d91b75f1428faccc573fcbf419a1: stopping tablet replica
I20260709 12:27:42.084491 29845 master.cc:584] Master@127.29.37.126:44053 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (268231 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:27:42.199395 29845 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.29.37.126:36231
I20260709 12:27:42.200546 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:42.205962  6700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:42.206028  6701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:42.207732  6703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:42.208876 29845 server_base.cc:1061] running on GCE node
I20260709 12:27:42.211884 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:42.212082 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:42.212245 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600062212210 us; error 0 us; skew 500 ppm
I20260709 12:27:42.212779 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:42.215427 29845 webserver.cc:533] Webserver started at http://127.29.37.126:46459/ using document root <none> and password file <none>
I20260709 12:27:42.215910 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:42.216097 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:42.216393 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:42.217613 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/master-0-root/instance:
uuid: "66ecd0b72fc14e0b8a155ba9063e4623"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.221808 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:27:42.225030  6708 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.225700 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:27:42.225978 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/master-0-root
uuid: "66ecd0b72fc14e0b8a155ba9063e4623"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.226282 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:42.259402 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:42.260620 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:42.299410 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.126:36231
I20260709 12:27:42.299492  6759 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.126:36231 every 8 connection(s)
I20260709 12:27:42.303582  6760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:42.314939  6760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623: Bootstrap starting.
I20260709 12:27:42.319796  6760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:42.323911  6760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623: No bootstrap required, opened a new log
I20260709 12:27:42.326144  6760 raft_consensus.cc:359] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ecd0b72fc14e0b8a155ba9063e4623" member_type: VOTER }
I20260709 12:27:42.326668  6760 raft_consensus.cc:385] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:42.326901  6760 raft_consensus.cc:740] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ecd0b72fc14e0b8a155ba9063e4623, State: Initialized, Role: FOLLOWER
I20260709 12:27:42.327483  6760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ecd0b72fc14e0b8a155ba9063e4623" member_type: VOTER }
I20260709 12:27:42.327967  6760 raft_consensus.cc:399] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:27:42.328262  6760 raft_consensus.cc:493] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:27:42.328539  6760 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:42.333117  6760 raft_consensus.cc:515] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ecd0b72fc14e0b8a155ba9063e4623" member_type: VOTER }
I20260709 12:27:42.333617  6760 leader_election.cc:304] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66ecd0b72fc14e0b8a155ba9063e4623; no voters: 
I20260709 12:27:42.334766  6760 leader_election.cc:290] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:27:42.335081  6763 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:42.336467  6763 raft_consensus.cc:697] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 1 LEADER]: Becoming Leader. State: Replica: 66ecd0b72fc14e0b8a155ba9063e4623, State: Running, Role: LEADER
I20260709 12:27:42.337153  6763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ecd0b72fc14e0b8a155ba9063e4623" member_type: VOTER }
I20260709 12:27:42.337798  6760 sys_catalog.cc:565] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:27:42.345906  6764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66ecd0b72fc14e0b8a155ba9063e4623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ecd0b72fc14e0b8a155ba9063e4623" member_type: VOTER } }
I20260709 12:27:42.346604  6764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [sys.catalog]: This master's current role is: LEADER
I20260709 12:27:42.346553  6765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66ecd0b72fc14e0b8a155ba9063e4623. Latest consensus state: current_term: 1 leader_uuid: "66ecd0b72fc14e0b8a155ba9063e4623" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ecd0b72fc14e0b8a155ba9063e4623" member_type: VOTER } }
I20260709 12:27:42.347221  6765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [sys.catalog]: This master's current role is: LEADER
I20260709 12:27:42.348762  6773 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:27:42.356325  6773 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:27:42.357264 29845 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:27:42.365249  6773 catalog_manager.cc:1383] Generated new cluster ID: d03324a2ee5942bdab83c31963ea2ce4
I20260709 12:27:42.365530  6773 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:27:42.387900  6773 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:27:42.389240  6773 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:27:42.402987  6773 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623: Generated new TSK 0
I20260709 12:27:42.403565  6773 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:27:42.423528 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:42.430994  6782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:42.430119  6781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:42.432842  6784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:42.434175 29845 server_base.cc:1061] running on GCE node
I20260709 12:27:42.437198 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:42.437429 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:42.437613 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600062437589 us; error 0 us; skew 500 ppm
I20260709 12:27:42.438195 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:42.440794 29845 webserver.cc:533] Webserver started at http://127.29.37.65:41905/ using document root <none> and password file <none>
I20260709 12:27:42.441310 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:42.441521 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:42.441823 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:42.443111 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/instance:
uuid: "38b3b964f82644c7b654e8042953a74e"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.447642 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:27:42.450878  6789 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.451553 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:27:42.451848 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "38b3b964f82644c7b654e8042953a74e"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.452140 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:42.484264 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:42.485481 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:42.487032 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:42.489290 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:42.489543 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.489780 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:42.489965 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.530808 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:45171
I20260709 12:27:42.530927  6851 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:45171 every 8 connection(s)
I20260709 12:27:42.536207 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:42.543732  6856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:42.544327  6857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:42.547746  6860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:42.548319 29845 server_base.cc:1061] running on GCE node
I20260709 12:27:42.551609 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:42.551853 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:42.552033 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600062552016 us; error 0 us; skew 500 ppm
I20260709 12:27:42.552590 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:42.554858  6852 heartbeater.cc:344] Connected to a master server at 127.29.37.126:36231
I20260709 12:27:42.555222  6852 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:42.555962  6852 heartbeater.cc:507] Master 127.29.37.126:36231 requested a full tablet report, sending...
I20260709 12:27:42.556082 29845 webserver.cc:533] Webserver started at http://127.29.37.66:36025/ using document root <none> and password file <none>
I20260709 12:27:42.556639 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:42.556849 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:42.557128 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:42.558148  6725 ts_manager.cc:194] Registered new tserver with Master: 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:42.558425 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/instance:
uuid: "99db41e57175426b9ceb11625b4455d9"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.560081  6725 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49552
I20260709 12:27:42.563198 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:27:42.566722  6864 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.567432 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:27:42.567703 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-1-root
uuid: "99db41e57175426b9ceb11625b4455d9"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.567935 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:42.584689 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:42.585827 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:42.587231 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:42.589346 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:42.589551 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.589798 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:42.589934 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.630034 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.66:34951
I20260709 12:27:42.630133  6926 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.66:34951 every 8 connection(s)
I20260709 12:27:42.635296 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:42.643199  6930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:42.645023  6931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:42.648152 29845 server_base.cc:1061] running on GCE node
W20260709 12:27:42.648222  6933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:42.649531 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:27:42.649549  6927 heartbeater.cc:344] Connected to a master server at 127.29.37.126:36231
W20260709 12:27:42.649938 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:42.650131  6927 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:42.650163 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600062650148 us; error 0 us; skew 500 ppm
I20260709 12:27:42.651029 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:42.651044  6927 heartbeater.cc:507] Master 127.29.37.126:36231 requested a full tablet report, sending...
I20260709 12:27:42.653491  6725 ts_manager.cc:194] Registered new tserver with Master: 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:42.654156 29845 webserver.cc:533] Webserver started at http://127.29.37.67:37649/ using document root <none> and password file <none>
I20260709 12:27:42.654831 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:42.655067 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:42.655370 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:42.655439  6725 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49564
I20260709 12:27:42.656879 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/instance:
uuid: "c78bd8036f7040b99af7c501182cbe21"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.661197 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:27:42.664489  6938 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.665226 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:27:42.665508 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-2-root
uuid: "c78bd8036f7040b99af7c501182cbe21"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.665786 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:42.681948 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:42.683254 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:42.684711 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:42.686868 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:42.687095 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.687337 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:42.687512 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.728327 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.67:41667
I20260709 12:27:42.728453  7000 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.67:41667 every 8 connection(s)
I20260709 12:27:42.733229 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:42.740669  7004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:42.745232  7005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:42.745416 29845 server_base.cc:1061] running on GCE node
W20260709 12:27:42.746006  7007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:42.746426  7001 heartbeater.cc:344] Connected to a master server at 127.29.37.126:36231
I20260709 12:27:42.746937  7001 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:42.747327 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:42.747604 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:42.747848 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600062747825 us; error 0 us; skew 500 ppm
I20260709 12:27:42.747876  7001 heartbeater.cc:507] Master 127.29.37.126:36231 requested a full tablet report, sending...
I20260709 12:27:42.748777 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:42.750362  6725 ts_manager.cc:194] Registered new tserver with Master: c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:42.751585 29845 webserver.cc:533] Webserver started at http://127.29.37.68:39483/ using document root <none> and password file <none>
I20260709 12:27:42.752040  6725 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49580
I20260709 12:27:42.752148 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:42.752514 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:42.752817 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:42.754457 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-3-root/instance:
uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.758826 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:27:42.762053  7012 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.762807 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:27:42.763113 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-3-root
uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:42.763393 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:42.777623 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:42.778863 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:42.780296 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:42.782485 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:42.782691 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.782946 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:42.783106 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:42.823559 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.68:41389
I20260709 12:27:42.823688  7074 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.68:41389 every 8 connection(s)
I20260709 12:27:42.838268  7075 heartbeater.cc:344] Connected to a master server at 127.29.37.126:36231
I20260709 12:27:42.838660  7075 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:42.839547  7075 heartbeater.cc:507] Master 127.29.37.126:36231 requested a full tablet report, sending...
I20260709 12:27:42.841398  6725 ts_manager.cc:194] Registered new tserver with Master: 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:42.841600 29845 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015006374s
I20260709 12:27:42.842849  6725 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49582
I20260709 12:27:42.867352  6725 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49592:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:27:43.111506  6889 tablet_service.cc:1511] Processing CreateTablet for tablet 7b68394604b04367a2e1169ce7623abc (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:27:43.113018  6889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b68394604b04367a2e1169ce7623abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.115383  6888 tablet_service.cc:1511] Processing CreateTablet for tablet a38ba9e3e56242779781f03654f3dfd3 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:27:43.117533  6890 tablet_service.cc:1511] Processing CreateTablet for tablet ea38fed35e6549b3917d485f9af88382 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:27:43.119411  6891 tablet_service.cc:1511] Processing CreateTablet for tablet 847f2ce68f10434cb350d63a6573b3bd (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:27:43.121721  6892 tablet_service.cc:1511] Processing CreateTablet for tablet b2429fdc971b4c048fbc5f3ce3b6bc2f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:27:43.127409  7036 tablet_service.cc:1511] Processing CreateTablet for tablet e962d48b16794d78ad1a367540885945 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:27:43.128870  7036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e962d48b16794d78ad1a367540885945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.131120  6966 tablet_service.cc:1511] Processing CreateTablet for tablet 847f2ce68f10434cb350d63a6573b3bd (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:27:43.131464  6964 tablet_service.cc:1511] Processing CreateTablet for tablet ea38fed35e6549b3917d485f9af88382 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:27:43.132596  6966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847f2ce68f10434cb350d63a6573b3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.135433  6961 tablet_service.cc:1511] Processing CreateTablet for tablet 052922f90cab431dba48fb4991dee3fb (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:27:43.133912  6962 tablet_service.cc:1511] Processing CreateTablet for tablet e962d48b16794d78ad1a367540885945 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:27:43.135988  6958 tablet_service.cc:1511] Processing CreateTablet for tablet b21f93c356074f488a5be203ff0b1988 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:27:43.138177  6960 tablet_service.cc:1511] Processing CreateTablet for tablet 0aa7e52e7c5b4039a78f136142693567 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:27:43.138504  6957 tablet_service.cc:1511] Processing CreateTablet for tablet f3ed0915d1da4514bc4c51eddfcf5e31 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:27:43.140271  7037 tablet_service.cc:1511] Processing CreateTablet for tablet a38ba9e3e56242779781f03654f3dfd3 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:27:43.140961  7038 tablet_service.cc:1511] Processing CreateTablet for tablet ea38fed35e6549b3917d485f9af88382 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:27:43.141695  7037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ba9e3e56242779781f03654f3dfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.143409  7039 tablet_service.cc:1511] Processing CreateTablet for tablet 251b19e52f0146aa9a9cfc985e2763b4 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:27:43.144513  7040 tablet_service.cc:1511] Processing CreateTablet for tablet b2429fdc971b4c048fbc5f3ce3b6bc2f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:27:43.146365  6959 tablet_service.cc:1511] Processing CreateTablet for tablet e7eee84ade114ccaa536076ae9b04ac2 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:27:43.146925  6963 tablet_service.cc:1511] Processing CreateTablet for tablet 7b68394604b04367a2e1169ce7623abc (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:27:43.148247  6965 tablet_service.cc:1511] Processing CreateTablet for tablet 251b19e52f0146aa9a9cfc985e2763b4 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:27:43.151013  6964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea38fed35e6549b3917d485f9af88382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.150665  7035 tablet_service.cc:1511] Processing CreateTablet for tablet 052922f90cab431dba48fb4991dee3fb (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:27:43.151425  7034 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2693326c6b49989473c3de8c70d40a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:27:43.152756  6888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ba9e3e56242779781f03654f3dfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.154366  6891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847f2ce68f10434cb350d63a6573b3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.155913  6892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2429fdc971b4c048fbc5f3ce3b6bc2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.156380  6887 tablet_service.cc:1511] Processing CreateTablet for tablet 052922f90cab431dba48fb4991dee3fb (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:27:43.158350  7033 tablet_service.cc:1511] Processing CreateTablet for tablet e7eee84ade114ccaa536076ae9b04ac2 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:27:43.158607  6886 tablet_service.cc:1511] Processing CreateTablet for tablet 0aa7e52e7c5b4039a78f136142693567 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:27:43.160027  7031 tablet_service.cc:1511] Processing CreateTablet for tablet 7c55f4a15fa641de92c4922bc65c06be (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:27:43.160749  7032 tablet_service.cc:1511] Processing CreateTablet for tablet b21f93c356074f488a5be203ff0b1988 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:27:43.161283  6885 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2693326c6b49989473c3de8c70d40a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:27:43.162528  6961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 052922f90cab431dba48fb4991dee3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.162966  6890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea38fed35e6549b3917d485f9af88382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.163952  6884 tablet_service.cc:1511] Processing CreateTablet for tablet b21f93c356074f488a5be203ff0b1988 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:27:43.165035  6962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e962d48b16794d78ad1a367540885945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.165293  6884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b21f93c356074f488a5be203ff0b1988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.167155  7038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea38fed35e6549b3917d485f9af88382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.167908  7032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b21f93c356074f488a5be203ff0b1988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.169839  7031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c55f4a15fa641de92c4922bc65c06be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.171600  7035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 052922f90cab431dba48fb4991dee3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.174549  6887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 052922f90cab431dba48fb4991dee3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.174751  7033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7eee84ade114ccaa536076ae9b04ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.178630  6886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa7e52e7c5b4039a78f136142693567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.179597  6817 tablet_service.cc:1511] Processing CreateTablet for tablet b2429fdc971b4c048fbc5f3ce3b6bc2f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:27:43.180943  6817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2429fdc971b4c048fbc5f3ce3b6bc2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.182706  7039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251b19e52f0146aa9a9cfc985e2763b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.186585  7040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2429fdc971b4c048fbc5f3ce3b6bc2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.186640  6885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2693326c6b49989473c3de8c70d40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.191977  6816 tablet_service.cc:1511] Processing CreateTablet for tablet 847f2ce68f10434cb350d63a6573b3bd (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:27:43.193414  6816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 847f2ce68f10434cb350d63a6573b3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.193889  6811 tablet_service.cc:1511] Processing CreateTablet for tablet 0aa7e52e7c5b4039a78f136142693567 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:27:43.195273  6811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa7e52e7c5b4039a78f136142693567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.195441  6815 tablet_service.cc:1511] Processing CreateTablet for tablet 251b19e52f0146aa9a9cfc985e2763b4 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:27:43.196781  6815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251b19e52f0146aa9a9cfc985e2763b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.199501  6883 tablet_service.cc:1511] Processing CreateTablet for tablet f3ed0915d1da4514bc4c51eddfcf5e31 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:27:43.200938  6813 tablet_service.cc:1511] Processing CreateTablet for tablet a38ba9e3e56242779781f03654f3dfd3 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:27:43.202354  6813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ba9e3e56242779781f03654f3dfd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.200973  6883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ed0915d1da4514bc4c51eddfcf5e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.203493  6812 tablet_service.cc:1511] Processing CreateTablet for tablet e962d48b16794d78ad1a367540885945 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:27:43.204895  6812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e962d48b16794d78ad1a367540885945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.199565  6814 tablet_service.cc:1511] Processing CreateTablet for tablet 7b68394604b04367a2e1169ce7623abc (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:27:43.207635  6814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b68394604b04367a2e1169ce7623abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.190573  7034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2693326c6b49989473c3de8c70d40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.220917  6963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b68394604b04367a2e1169ce7623abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.226954  6957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ed0915d1da4514bc4c51eddfcf5e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.230885  6959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7eee84ade114ccaa536076ae9b04ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.231716  6965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 251b19e52f0146aa9a9cfc985e2763b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.234565  6960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aa7e52e7c5b4039a78f136142693567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.238547  6958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b21f93c356074f488a5be203ff0b1988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.259814  6810 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2693326c6b49989473c3de8c70d40a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:27:43.261235  6810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2693326c6b49989473c3de8c70d40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.270191  6884 tablet_service.cc:1511] Processing CreateTablet for tablet 7c55f4a15fa641de92c4922bc65c06be (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:27:43.293009  6884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c55f4a15fa641de92c4922bc65c06be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.272294  6887 tablet_service.cc:1511] Processing CreateTablet for tablet b678720774394be3899ea14ff99628da (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:27:43.295745  6887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b678720774394be3899ea14ff99628da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.283720  6809 tablet_service.cc:1511] Processing CreateTablet for tablet e7eee84ade114ccaa536076ae9b04ac2 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:27:43.298748  6888 tablet_service.cc:1511] Processing CreateTablet for tablet 02d86f9e2d5d4ca195483d90646330a4 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:27:43.299935  6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7eee84ade114ccaa536076ae9b04ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.300393  6888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d86f9e2d5d4ca195483d90646330a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.285328  6808 tablet_service.cc:1511] Processing CreateTablet for tablet f3ed0915d1da4514bc4c51eddfcf5e31 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:27:43.304147  6808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ed0915d1da4514bc4c51eddfcf5e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.273633  6962 tablet_service.cc:1511] Processing CreateTablet for tablet 9875aff5560f4adc82ed2881b0719c77 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:27:43.311739  6962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9875aff5560f4adc82ed2881b0719c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.319439  6957 tablet_service.cc:1511] Processing CreateTablet for tablet b678720774394be3899ea14ff99628da (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:27:43.320879  6957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b678720774394be3899ea14ff99628da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.291582  6883 tablet_service.cc:1511] Processing CreateTablet for tablet afde26a33c5c45cfb6b2087601e4ce01 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:27:43.331594  6883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afde26a33c5c45cfb6b2087601e4ce01. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:43.349516  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:43.350391  6965 tablet_service.cc:1511] Processing CreateTablet for tablet 02d86f9e2d5d4ca195483d90646330a4 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:27:43.353585  6964 tablet_service.cc:1511] Processing CreateTablet for tablet d82cf9bc8bd54dc08a7e03270cacde99 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:27:43.354003  6965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d86f9e2d5d4ca195483d90646330a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.363746  7037 tablet_service.cc:1511] Processing CreateTablet for tablet 9875aff5560f4adc82ed2881b0719c77 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:27:43.345738  6886 tablet_service.cc:1511] Processing CreateTablet for tablet 75c82b1c500f4214aef5d8ed2567f780 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:27:43.367483  7038 tablet_service.cc:1511] Processing CreateTablet for tablet afde26a33c5c45cfb6b2087601e4ce01 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:27:43.367859  7033 tablet_service.cc:1511] Processing CreateTablet for tablet b678720774394be3899ea14ff99628da (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:27:43.374500  6964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82cf9bc8bd54dc08a7e03270cacde99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.374874  6886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75c82b1c500f4214aef5d8ed2567f780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.375257  7094 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:43.377462  6817 tablet_service.cc:1511] Processing CreateTablet for tablet 46b2ca102c7b4575a89399cac44ee866 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:27:43.378793  6817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b2ca102c7b4575a89399cac44ee866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.379434  6966 tablet_service.cc:1511] Processing CreateTablet for tablet 04e9253071944b29a6116b8d9f64ef3a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:27:43.380725  6966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04e9253071944b29a6116b8d9f64ef3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.386338  7096 tablet_bootstrap.cc:492] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:43.386739  7033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b678720774394be3899ea14ff99628da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.386924  7095 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:43.393572  6958 tablet_service.cc:1511] Processing CreateTablet for tablet 75c82b1c500f4214aef5d8ed2567f780 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:27:43.394790  7037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9875aff5560f4adc82ed2881b0719c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.396349  6810 tablet_service.cc:1511] Processing CreateTablet for tablet 9875aff5560f4adc82ed2881b0719c77 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:27:43.425676  6810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9875aff5560f4adc82ed2881b0719c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.430820  6814 tablet_service.cc:1511] Processing CreateTablet for tablet afde26a33c5c45cfb6b2087601e4ce01 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:27:43.398866  6811 tablet_service.cc:1511] Processing CreateTablet for tablet 7c55f4a15fa641de92c4922bc65c06be (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:27:43.441257  6811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c55f4a15fa641de92c4922bc65c06be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.441074  6891 tablet_service.cc:1511] Processing CreateTablet for tablet d82cf9bc8bd54dc08a7e03270cacde99 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:27:43.442420  6891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82cf9bc8bd54dc08a7e03270cacde99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.407351  7032 tablet_service.cc:1511] Processing CreateTablet for tablet 75c82b1c500f4214aef5d8ed2567f780 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:27:43.447011  7038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afde26a33c5c45cfb6b2087601e4ce01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.447260  6814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afde26a33c5c45cfb6b2087601e4ce01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.409578  6960 tablet_service.cc:1511] Processing CreateTablet for tablet 46b2ca102c7b4575a89399cac44ee866 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:27:43.451071  6960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b2ca102c7b4575a89399cac44ee866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.451423  7032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75c82b1c500f4214aef5d8ed2567f780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.458787  7036 tablet_service.cc:1511] Processing CreateTablet for tablet 04e9253071944b29a6116b8d9f64ef3a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:27:43.459263  6809 tablet_service.cc:1511] Processing CreateTablet for tablet 8c77ef2e07ff418b996e1c72415228d6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:27:43.460122  7036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04e9253071944b29a6116b8d9f64ef3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.460740  6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c77ef2e07ff418b996e1c72415228d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.461795  7094 tablet_bootstrap.cc:654] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:43.464946  6959 tablet_service.cc:1511] Processing CreateTablet for tablet bfbf51a7748b461686ff7db762bba509 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:27:43.416414  7096 tablet_bootstrap.cc:654] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:43.466288  6959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfbf51a7748b461686ff7db762bba509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.466392  6808 tablet_service.cc:1511] Processing CreateTablet for tablet 93204a5a0d0742ad9cd21a8a280af392 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:27:43.436403  6892 tablet_service.cc:1511] Processing CreateTablet for tablet 46b2ca102c7b4575a89399cac44ee866 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:27:43.400346  6813 tablet_service.cc:1511] Processing CreateTablet for tablet 02d86f9e2d5d4ca195483d90646330a4 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:27:43.439116  6885 tablet_service.cc:1511] Processing CreateTablet for tablet 8c77ef2e07ff418b996e1c72415228d6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:27:43.439956  6889 tablet_service.cc:1511] Processing CreateTablet for tablet bfbf51a7748b461686ff7db762bba509 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:27:43.472946  6958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75c82b1c500f4214aef5d8ed2567f780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.489624  6892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46b2ca102c7b4575a89399cac44ee866. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.489845  6890 tablet_service.cc:1511] Processing CreateTablet for tablet c2acfad3040b40b69d4e0262179cdf6e (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:27:43.491210  6890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2acfad3040b40b69d4e0262179cdf6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.493920  6812 tablet_service.cc:1511] Processing CreateTablet for tablet 04e9253071944b29a6116b8d9f64ef3a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:27:43.495060  6885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c77ef2e07ff418b996e1c72415228d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.495929  7097 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:43.497859  6964 tablet_service.cc:1511] Processing CreateTablet for tablet 86c78f0ec1af4414a7e52c69f19b6c27 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:27:43.498834  6889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfbf51a7748b461686ff7db762bba509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.499174  6964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c78f0ec1af4414a7e52c69f19b6c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.472635  6888 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc898afa9a94274850432d379c5203e (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:27:43.503091  6888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc898afa9a94274850432d379c5203e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.433730  7031 tablet_service.cc:1511] Processing CreateTablet for tablet 8c77ef2e07ff418b996e1c72415228d6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:27:43.481210  7040 tablet_service.cc:1511] Processing CreateTablet for tablet 93204a5a0d0742ad9cd21a8a280af392 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:27:43.505925  6963 tablet_service.cc:1511] Processing CreateTablet for tablet c2acfad3040b40b69d4e0262179cdf6e (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:27:43.507207  7040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93204a5a0d0742ad9cd21a8a280af392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.477015  7095 tablet_bootstrap.cc:654] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:43.510836  6808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93204a5a0d0742ad9cd21a8a280af392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.511936  6812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04e9253071944b29a6116b8d9f64ef3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.517158  7033 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc898afa9a94274850432d379c5203e (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:27:43.517823  6813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02d86f9e2d5d4ca195483d90646330a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.507234  6963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2acfad3040b40b69d4e0262179cdf6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.477996  6961 tablet_service.cc:1511] Processing CreateTablet for tablet 93204a5a0d0742ad9cd21a8a280af392 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:27:43.516533  7034 tablet_service.cc:1511] Processing CreateTablet for tablet bfbf51a7748b461686ff7db762bba509 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:27:43.530966  6961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93204a5a0d0742ad9cd21a8a280af392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.540095  7034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfbf51a7748b461686ff7db762bba509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.549336  6811 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc898afa9a94274850432d379c5203e (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:27:43.550843  6811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc898afa9a94274850432d379c5203e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.553277  6962 tablet_service.cc:1511] Processing CreateTablet for tablet 122d63cf583043458b719df07a6a694d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:27:43.570729  6962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 122d63cf583043458b719df07a6a694d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.559019  6885 tablet_service.cc:1511] Processing CreateTablet for tablet 36d0e6da2d6d4fa6abb2757dff7de498 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:27:43.562858  6852 heartbeater.cc:499] Master 127.29.37.126:36231 was elected leader, sending a full tablet report...
I20260709 12:27:43.574263  6965 tablet_service.cc:1511] Processing CreateTablet for tablet a85b20398cd74f5888b96dc53ab9179b (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:27:43.575517  6965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85b20398cd74f5888b96dc53ab9179b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.567868  6808 tablet_service.cc:1511] Processing CreateTablet for tablet 86c78f0ec1af4414a7e52c69f19b6c27 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:27:43.557478  6887 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac46831fcf945d194fd7677ada23856 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:27:43.570595  6817 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac46831fcf945d194fd7677ada23856 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:27:43.596076  7039 tablet_service.cc:1511] Processing CreateTablet for tablet 86c78f0ec1af4414a7e52c69f19b6c27 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:27:43.597330  7096 tablet_bootstrap.cc:492] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:43.597878  7096 ts_tablet_manager.cc:1403] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.254s	user 0.004s	sys 0.010s
I20260709 12:27:43.590303  6816 tablet_service.cc:1511] Processing CreateTablet for tablet d82cf9bc8bd54dc08a7e03270cacde99 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:27:43.598944  7033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc898afa9a94274850432d379c5203e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.600090  6887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac46831fcf945d194fd7677ada23856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.601015  7096 raft_consensus.cc:359] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:43.601783  7096 raft_consensus.cc:385] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:43.602680  7097 tablet_bootstrap.cc:654] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:43.603508  7039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c78f0ec1af4414a7e52c69f19b6c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.602757  6885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d0e6da2d6d4fa6abb2757dff7de498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.607108  6957 tablet_service.cc:1511] Processing CreateTablet for tablet e12aacff6f754dfaa669e9651cce7dd9 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:27:43.608484  6957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12aacff6f754dfaa669e9651cce7dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.602056  7096 raft_consensus.cc:740] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:43.627477  6816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d82cf9bc8bd54dc08a7e03270cacde99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.615399  6886 tablet_service.cc:1511] Processing CreateTablet for tablet a85b20398cd74f5888b96dc53ab9179b (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:27:43.639492  7095 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:43.632890  6810 tablet_service.cc:1511] Processing CreateTablet for tablet 36d0e6da2d6d4fa6abb2757dff7de498 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:27:43.647917  7095 ts_tablet_manager.cc:1403] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.261s	user 0.006s	sys 0.009s
I20260709 12:27:43.648615  7031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c77ef2e07ff418b996e1c72415228d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.651016  7095 raft_consensus.cc:359] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:43.651762  7095 raft_consensus.cc:385] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:43.652057  7095 raft_consensus.cc:740] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:43.653561  7095 consensus_queue.cc:260] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:43.659492  6927 heartbeater.cc:499] Master 127.29.37.126:36231 was elected leader, sending a full tablet report...
I20260709 12:27:43.654789  7038 tablet_service.cc:1511] Processing CreateTablet for tablet 122d63cf583043458b719df07a6a694d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:27:43.663679  7038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 122d63cf583043458b719df07a6a694d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.663857  6883 tablet_service.cc:1511] Processing CreateTablet for tablet f0698605f5af41fcbc332b4dc8857606 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:27:43.640645  7096 consensus_queue.cc:260] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:43.666375  6891 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab076c84814416690df2fe770a98aa0 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:27:43.667609  6815 tablet_service.cc:1511] Processing CreateTablet for tablet 122d63cf583043458b719df07a6a694d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:27:43.654737  6808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c78f0ec1af4414a7e52c69f19b6c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.677066  7094 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:43.677659  7094 ts_tablet_manager.cc:1403] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.303s	user 0.021s	sys 0.008s
I20260709 12:27:43.678308  7035 tablet_service.cc:1511] Processing CreateTablet for tablet c2acfad3040b40b69d4e0262179cdf6e (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:27:43.679646  7035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2acfad3040b40b69d4e0262179cdf6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.680092  6814 tablet_service.cc:1511] Processing CreateTablet for tablet e12aacff6f754dfaa669e9651cce7dd9 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:27:43.639736  6966 tablet_service.cc:1511] Processing CreateTablet for tablet 4ac46831fcf945d194fd7677ada23856 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:27:43.683164  6966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac46831fcf945d194fd7677ada23856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.684820  6888 tablet_service.cc:1511] Processing CreateTablet for tablet 5257ef3bead24ca48d68d7b8415e116f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:27:43.685930  7032 tablet_service.cc:1511] Processing CreateTablet for tablet 36d0e6da2d6d4fa6abb2757dff7de498 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:27:43.686143  6888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5257ef3bead24ca48d68d7b8415e116f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.686786  6884 tablet_service.cc:1511] Processing CreateTablet for tablet e12aacff6f754dfaa669e9651cce7dd9 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:27:43.687443  7032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d0e6da2d6d4fa6abb2757dff7de498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.688022  6884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12aacff6f754dfaa669e9651cce7dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.689980  7097 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:43.690503  6883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0698605f5af41fcbc332b4dc8857606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.690980  7097 ts_tablet_manager.cc:1403] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.196s	user 0.011s	sys 0.006s
I20260709 12:27:43.692781  6813 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab076c84814416690df2fe770a98aa0 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:27:43.695223  6809 tablet_service.cc:1511] Processing CreateTablet for tablet 8203d585d06840daadbcfbb32fb2f822 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:27:43.697507  6814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e12aacff6f754dfaa669e9651cce7dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.745349  6817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ac46831fcf945d194fd7677ada23856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.698061  6886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85b20398cd74f5888b96dc53ab9179b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.745966  7097 raft_consensus.cc:359] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:43.704782  6960 tablet_service.cc:1511] Processing CreateTablet for tablet f0698605f5af41fcbc332b4dc8857606 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:27:43.708686  6961 tablet_service.cc:1511] Processing CreateTablet for tablet 5257ef3bead24ca48d68d7b8415e116f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:27:43.709043  6890 tablet_service.cc:1511] Processing CreateTablet for tablet e11f6b3dd5fe466d984f796948818c0d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:27:43.690815  7037 tablet_service.cc:1511] Processing CreateTablet for tablet a85b20398cd74f5888b96dc53ab9179b (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:27:43.718484  7096 ts_tablet_manager.cc:1434] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.120s	user 0.008s	sys 0.002s
I20260709 12:27:43.719954  6964 tablet_service.cc:1511] Processing CreateTablet for tablet efdd38fdcb5f4e2198f1180e3fe51a10 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:27:43.731532  7094 raft_consensus.cc:359] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:43.734333  7075 heartbeater.cc:499] Master 127.29.37.126:36231 was elected leader, sending a full tablet report...
I20260709 12:27:43.762398  6810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d0e6da2d6d4fa6abb2757dff7de498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.763144  6813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab076c84814416690df2fe770a98aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.759160  6891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab076c84814416690df2fe770a98aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.765192  7094 raft_consensus.cc:385] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:43.765558  7094 raft_consensus.cc:740] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:43.699204  6958 tablet_service.cc:1511] Processing CreateTablet for tablet 8203d585d06840daadbcfbb32fb2f822 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:27:43.767231  6890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e11f6b3dd5fe466d984f796948818c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.753752  7103 raft_consensus.cc:493] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.754859  7001 heartbeater.cc:499] Master 127.29.37.126:36231 was elected leader, sending a full tablet report...
I20260709 12:27:43.769296  7103 raft_consensus.cc:515] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:43.765297  7096 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:43.735555  6889 tablet_service.cc:1511] Processing CreateTablet for tablet a2d766dbe21d42a0ab36d005d89289c9 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:27:43.782392  6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8203d585d06840daadbcfbb32fb2f822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.785248  6958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8203d585d06840daadbcfbb32fb2f822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.787386  6964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efdd38fdcb5f4e2198f1180e3fe51a10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.794780  6889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d766dbe21d42a0ab36d005d89289c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.748956  7097 raft_consensus.cc:385] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:43.816458  6816 tablet_service.cc:1511] Processing CreateTablet for tablet 5257ef3bead24ca48d68d7b8415e116f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:27:43.822362  6961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5257ef3bead24ca48d68d7b8415e116f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:43.828474  7076 tablet.cc:2406] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:27:43.831126  6813 tablet_service.cc:1511] Processing CreateTablet for tablet e11f6b3dd5fe466d984f796948818c0d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:27:43.832213  7037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85b20398cd74f5888b96dc53ab9179b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.800169  7094 consensus_queue.cc:260] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:43.834951  6959 tablet_service.cc:1511] Processing CreateTablet for tablet a2d766dbe21d42a0ab36d005d89289c9 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:27:43.835820  6815 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 122d63cf583043458b719df07a6a694d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.839602  7096 tablet_bootstrap.cc:654] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:43.851910  6812 tablet_service.cc:1511] Processing CreateTablet for tablet efdd38fdcb5f4e2198f1180e3fe51a10 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:27:43.857419  7031 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab076c84814416690df2fe770a98aa0 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:27:43.863284  7095 ts_tablet_manager.cc:1434] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.215s	user 0.008s	sys 0.016s
I20260709 12:27:43.869539  6887 tablet_service.cc:1511] Processing CreateTablet for tablet d2fcd7006bc840a0bb5ea27365793f86 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:27:43.870612  6963 tablet_service.cc:1511] Processing CreateTablet for tablet d2fcd7006bc840a0bb5ea27365793f86 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:27:43.812595  7097 raft_consensus.cc:740] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:43.798661  7103 leader_election.cc:290] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:43.892340  6960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0698605f5af41fcbc332b4dc8857606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.906064  6816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5257ef3bead24ca48d68d7b8415e116f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.926982  6963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2fcd7006bc840a0bb5ea27365793f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.934139  6959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d766dbe21d42a0ab36d005d89289c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.934508  6813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e11f6b3dd5fe466d984f796948818c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.946581  6812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efdd38fdcb5f4e2198f1180e3fe51a10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.953271  7095 tablet_bootstrap.cc:492] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:43.910141  6887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2fcd7006bc840a0bb5ea27365793f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.955054  6811 tablet_service.cc:1511] Processing CreateTablet for tablet d2fcd7006bc840a0bb5ea27365793f86 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:27:43.964135  6811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2fcd7006bc840a0bb5ea27365793f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.956830  7094 ts_tablet_manager.cc:1434] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.279s	user 0.007s	sys 0.022s
I20260709 12:27:43.956048  7034 tablet_service.cc:1511] Processing CreateTablet for tablet 8203d585d06840daadbcfbb32fb2f822 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:27:43.971299  7097 consensus_queue.cc:260] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:43.972298  7094 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:43.981343  7033 tablet_service.cc:1511] Processing CreateTablet for tablet a2d766dbe21d42a0ab36d005d89289c9 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:27:43.981393  7036 tablet_service.cc:1511] Processing CreateTablet for tablet efdd38fdcb5f4e2198f1180e3fe51a10 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:27:43.982906  7033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2d766dbe21d42a0ab36d005d89289c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.984380  7036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efdd38fdcb5f4e2198f1180e3fe51a10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.985056  7038 tablet_service.cc:1511] Processing CreateTablet for tablet e381f83716b64b7982f5f0c9c63d348f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:27:43.986407  7038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e381f83716b64b7982f5f0c9c63d348f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:43.965714  7039 tablet_service.cc:1511] Processing CreateTablet for tablet f0698605f5af41fcbc332b4dc8857606 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:27:44.024827  7096 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:44.025393  7096 ts_tablet_manager.cc:1403] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.260s	user 0.007s	sys 0.014s
I20260709 12:27:44.027716  7040 tablet_service.cc:1511] Processing CreateTablet for tablet e11f6b3dd5fe466d984f796948818c0d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:27:44.030838  6817 tablet_service.cc:1511] Processing CreateTablet for tablet e381f83716b64b7982f5f0c9c63d348f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:27:44.032133  6817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e381f83716b64b7982f5f0c9c63d348f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.032747  7096 raft_consensus.cc:359] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.032855  6890 tablet_service.cc:1511] Processing CreateTablet for tablet e381f83716b64b7982f5f0c9c63d348f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:27:44.046751  6890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e381f83716b64b7982f5f0c9c63d348f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.046882  6961 tablet_service.cc:1511] Processing CreateTablet for tablet 4d22d1742e5946998079e448d9ab49ef (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:27:44.048265  6961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d22d1742e5946998079e448d9ab49ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.039582  6957 tablet_service.cc:1511] Processing CreateTablet for tablet cb45dbc587b34cb7843e48180e0219c3 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:27:44.051076  6957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb45dbc587b34cb7843e48180e0219c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.054389  7096 raft_consensus.cc:385] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.054786  7096 raft_consensus.cc:740] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.055548  7096 consensus_queue.cc:260] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.045074  7108 raft_consensus.cc:493] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.045322  6887 tablet_service.cc:1511] Processing CreateTablet for tablet 4d22d1742e5946998079e448d9ab49ef (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:27:44.074095  6887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d22d1742e5946998079e448d9ab49ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.074157  6809 tablet_service.cc:1511] Processing CreateTablet for tablet cb45dbc587b34cb7843e48180e0219c3 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:27:44.075587  6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb45dbc587b34cb7843e48180e0219c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.059401  7094 tablet_bootstrap.cc:654] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.078387  7108 raft_consensus.cc:515] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.081231  6883 tablet_service.cc:1511] Processing CreateTablet for tablet 718060ceb1df4264ac14b8e4c8f9c007 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:27:44.082554  6883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718060ceb1df4264ac14b8e4c8f9c007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.082903  7108 leader_election.cc:290] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:44.068514  7035 tablet_service.cc:1511] Processing CreateTablet for tablet 4d22d1742e5946998079e448d9ab49ef (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:27:44.070387  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b21f93c356074f488a5be203ff0b1988" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:44.071437  7034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8203d585d06840daadbcfbb32fb2f822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.044431  7095 tablet_bootstrap.cc:654] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.092262  7096 ts_tablet_manager.cc:1434] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.001s
I20260709 12:27:44.111016  7096 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:44.111033  7031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab076c84814416690df2fe770a98aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.111837  6813 tablet_service.cc:1511] Processing CreateTablet for tablet 718060ceb1df4264ac14b8e4c8f9c007 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:27:44.113111  6813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718060ceb1df4264ac14b8e4c8f9c007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.114883  7035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d22d1742e5946998079e448d9ab49ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.118505  7039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0698605f5af41fcbc332b4dc8857606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.122500  7040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e11f6b3dd5fe466d984f796948818c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.128888  6811 tablet_service.cc:1511] Processing CreateTablet for tablet cd4c913d08ea42b09038a2a80ecfa3b6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:27:44.129336  7097 ts_tablet_manager.cc:1434] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.386s	user 0.016s	sys 0.025s
I20260709 12:27:44.130256  6811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4c913d08ea42b09038a2a80ecfa3b6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:44.096865  7016 leader_election.cc:343] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.139168  7096 tablet_bootstrap.cc:654] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.140957  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b21f93c356074f488a5be203ff0b1988" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:44.141100  6961 tablet_service.cc:1511] Processing CreateTablet for tablet 718060ceb1df4264ac14b8e4c8f9c007 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:27:44.142714  7097 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:44.142999  7102 raft_consensus.cc:493] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.143476  7102 raft_consensus.cc:515] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.126988  6889 tablet_service.cc:1511] Processing CreateTablet for tablet cd4c913d08ea42b09038a2a80ecfa3b6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:27:44.150877  6889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4c913d08ea42b09038a2a80ecfa3b6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:44.153723  7013 leader_election.cc:343] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.157936  7097 tablet_bootstrap.cc:654] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.146934  7032 tablet_service.cc:1511] Processing CreateTablet for tablet cd4c913d08ea42b09038a2a80ecfa3b6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:27:44.158931  7036 tablet_service.cc:1511] Processing CreateTablet for tablet cb45dbc587b34cb7843e48180e0219c3 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
W20260709 12:27:44.147903  7016 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 55107.5us
I20260709 12:27:44.165974  6961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718060ceb1df4264ac14b8e4c8f9c007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.166719  7032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4c913d08ea42b09038a2a80ecfa3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.170341  7036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb45dbc587b34cb7843e48180e0219c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.175300  7102 leader_election.cc:290] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:44.196244  7037 tablet_service.cc:1511] Processing CreateTablet for tablet 70fa5eb82f684b449d82176587520e4d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:27:44.197611  7037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70fa5eb82f684b449d82176587520e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.199465  6959 tablet_service.cc:1511] Processing CreateTablet for tablet 70fa5eb82f684b449d82176587520e4d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:27:44.200706  6959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70fa5eb82f684b449d82176587520e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.165233  7013 leader_election.cc:304] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:44.223289  7103 raft_consensus.cc:2749] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:44.224272  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 81817.4us
I20260709 12:27:44.231241  7031 tablet_service.cc:1511] Processing CreateTablet for tablet 2dfa9e71770a4e3dac4348d84a9cc2d6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:27:44.231464  6887 tablet_service.cc:1511] Processing CreateTablet for tablet 2dfa9e71770a4e3dac4348d84a9cc2d6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:27:44.232575  7031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfa9e71770a4e3dac4348d84a9cc2d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.232762  6887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfa9e71770a4e3dac4348d84a9cc2d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.235069  7111 raft_consensus.cc:493] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.235839  7095 tablet_bootstrap.cc:492] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:44.235575  7111 raft_consensus.cc:515] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.239162  6812 tablet_service.cc:1511] Processing CreateTablet for tablet 70fa5eb82f684b449d82176587520e4d (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:27:44.248488  6812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70fa5eb82f684b449d82176587520e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.248799  7111 leader_election.cc:290] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:44.255482  7095 ts_tablet_manager.cc:1403] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.330s	user 0.035s	sys 0.007s
I20260709 12:27:44.262507  6957 tablet_service.cc:1511] Processing CreateTablet for tablet 2dfa9e71770a4e3dac4348d84a9cc2d6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:27:44.263309  6890 tablet_service.cc:1511] Processing CreateTablet for tablet 580547b3f100485eba38ef98536bbc93 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:27:44.267869  7096 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:44.275038  6890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580547b3f100485eba38ef98536bbc93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.275629  7096 ts_tablet_manager.cc:1403] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.165s	user 0.011s	sys 0.012s
I20260709 12:27:44.281304  7096 raft_consensus.cc:359] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.281538  6809 tablet_service.cc:1511] Processing CreateTablet for tablet 5b8fc940ac6a48f0ab40f16b9a7b9ca6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:27:44.282883  6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b8fc940ac6a48f0ab40f16b9a7b9ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.284313  7103 raft_consensus.cc:493] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.284662  7095 raft_consensus.cc:359] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.284804  7103 raft_consensus.cc:515] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.285431  7095 raft_consensus.cc:385] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.285738  7095 raft_consensus.cc:740] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.287163  7103 leader_election.cc:290] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:44.288820  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:44.289577  7035 tablet_service.cc:1511] Processing CreateTablet for tablet 5b8fc940ac6a48f0ab40f16b9a7b9ca6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:27:44.290985  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847f2ce68f10434cb350d63a6573b3bd" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:44.292754  6939 leader_election.cc:343] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.294950  6816 tablet_service.cc:1511] Processing CreateTablet for tablet 580547b3f100485eba38ef98536bbc93 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:27:44.296257  6816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580547b3f100485eba38ef98536bbc93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.291255  6957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfa9e71770a4e3dac4348d84a9cc2d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.282014  7096 raft_consensus.cc:385] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.298511  7096 raft_consensus.cc:740] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.299541  7096 consensus_queue.cc:260] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.290982  7035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b8fc940ac6a48f0ab40f16b9a7b9ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.302963  7095 consensus_queue.cc:260] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
W20260709 12:27:44.309171  7016 leader_election.cc:343] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.326236  7096 ts_tablet_manager.cc:1434] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 12:27:44.342202  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847f2ce68f10434cb350d63a6573b3bd" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:44.343009  6902 raft_consensus.cc:2468] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:44.343570  7096 tablet_bootstrap.cc:492] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:44.346768  6942 leader_election.cc:304] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:44.349261  7108 raft_consensus.cc:2804] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:44.349677  7108 raft_consensus.cc:493] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.350011  7108 raft_consensus.cc:3060] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.352283  7094 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:44.352843  7094 ts_tablet_manager.cc:1403] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.381s	user 0.012s	sys 0.010s
W20260709 12:27:44.358522  6712 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 53254.2us
W20260709 12:27:44.359604  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:44.362242  7095 ts_tablet_manager.cc:1434] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.101s	user 0.006s	sys 0.000s
I20260709 12:27:44.363372  7095 tablet_bootstrap.cc:492] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:44.365000  7108 raft_consensus.cc:515] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.368240  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847f2ce68f10434cb350d63a6573b3bd" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
W20260709 12:27:44.369801  6939 leader_election.cc:343] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.370905  7097 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:44.371443  7097 ts_tablet_manager.cc:1403] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.229s	user 0.012s	sys 0.010s
I20260709 12:27:44.372920  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:44.374519  6793 leader_election.cc:343] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.375320  6961 tablet_service.cc:1511] Processing CreateTablet for tablet 580547b3f100485eba38ef98536bbc93 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:27:44.376734  6961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 580547b3f100485eba38ef98536bbc93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.383261  7036 tablet_service.cc:1511] Processing CreateTablet for tablet c92afb4b5d234fd48fe0f24ec262964a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:27:44.386550  7036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92afb4b5d234fd48fe0f24ec262964a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.388042  6963 tablet_service.cc:1511] Processing CreateTablet for tablet c92afb4b5d234fd48fe0f24ec262964a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:27:44.399895  6963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92afb4b5d234fd48fe0f24ec262964a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.401352  6809 tablet_service.cc:1511] Processing CreateTablet for tablet 22a6861b97e443898d98ede72c787513 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:27:44.389952  6960 tablet_service.cc:1511] Processing CreateTablet for tablet 864b9e85f77d4ecc8726f6d624b5780c (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:27:44.402752  6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22a6861b97e443898d98ede72c787513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.403100  6960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864b9e85f77d4ecc8726f6d624b5780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.390467  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:44.391340  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:44.392817  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847f2ce68f10434cb350d63a6573b3bd" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:44.407830  6902 raft_consensus.cc:3060] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.409195  6891 tablet_service.cc:1511] Processing CreateTablet for tablet 22a6861b97e443898d98ede72c787513 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:27:44.410660  6891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22a6861b97e443898d98ede72c787513. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:44.412240  7013 leader_election.cc:343] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.412768  7013 leader_election.cc:304] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:44.413991  7103 raft_consensus.cc:2749] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:44.415689  7038 tablet_service.cc:1511] Processing CreateTablet for tablet 864b9e85f77d4ecc8726f6d624b5780c (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:27:44.417203  7038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864b9e85f77d4ecc8726f6d624b5780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.393442  7096 tablet_bootstrap.cc:654] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.394590  7108 leader_election.cc:290] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:44.395057  6883 tablet_service.cc:1511] Processing CreateTablet for tablet 5b8fc940ac6a48f0ab40f16b9a7b9ca6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:27:44.395622  6889 tablet_service.cc:1511] Processing CreateTablet for tablet 864b9e85f77d4ecc8726f6d624b5780c (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:27:44.395900  6813 tablet_service.cc:1511] Processing CreateTablet for tablet c92afb4b5d234fd48fe0f24ec262964a (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:27:44.427367  6883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b8fc940ac6a48f0ab40f16b9a7b9ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.392737  7097 raft_consensus.cc:359] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.428145  7097 raft_consensus.cc:385] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.428309  6889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864b9e85f77d4ecc8726f6d624b5780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.428592  7097 raft_consensus.cc:740] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.429613  7097 consensus_queue.cc:260] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.435398  7097 ts_tablet_manager.cc:1434] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.000s
I20260709 12:27:44.436537  7097 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:44.427517  6813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92afb4b5d234fd48fe0f24ec262964a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.437109  6902 raft_consensus.cc:2468] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:44.438490  6942 leader_election.cc:304] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:44.439301  7108 raft_consensus.cc:2804] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:44.440752  7108 raft_consensus.cc:697] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:44.441727  7108 consensus_queue.cc:237] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.443166  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847f2ce68f10434cb350d63a6573b3bd" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:44.450273  6865 leader_election.cc:343] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.450978  7094 raft_consensus.cc:359] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.451710  7094 raft_consensus.cc:385] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.452008  7094 raft_consensus.cc:740] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
W20260709 12:27:44.453140  6790 leader_election.cc:343] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.453614  6790 leader_election.cc:304] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:44.454754  7111 raft_consensus.cc:2749] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:44.457764  6963 tablet_service.cc:1511] Processing CreateTablet for tablet 22a6861b97e443898d98ede72c787513 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:27:44.459131  6963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22a6861b97e443898d98ede72c787513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.461112  7036 tablet_service.cc:1511] Processing CreateTablet for tablet f932aaa29a8443d4b251dea7c9fc0f54 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:27:44.462522  7036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f932aaa29a8443d4b251dea7c9fc0f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.462146  7094 consensus_queue.cc:260] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.464823  7094 ts_tablet_manager.cc:1434] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.112s	user 0.008s	sys 0.000s
I20260709 12:27:44.468709  7094 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:44.469372  6723 catalog_manager.cc:5697] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:44.472016  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847f2ce68f10434cb350d63a6573b3bd" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:44.473673  6865 leader_election.cc:304] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:44.474836  7102 raft_consensus.cc:2749] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:44.475342  7096 tablet_bootstrap.cc:492] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:44.483165  7096 ts_tablet_manager.cc:1403] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.140s	user 0.013s	sys 0.000s
I20260709 12:27:44.485146  7094 tablet_bootstrap.cc:654] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.493568  7102 raft_consensus.cc:493] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.494073  7102 raft_consensus.cc:515] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.497443  7094 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:44.485865  7097 tablet_bootstrap.cc:654] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.488458  7096 raft_consensus.cc:359] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.479959  7095 tablet_bootstrap.cc:654] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.501332  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b21f93c356074f488a5be203ff0b1988" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:44.502401  7096 raft_consensus.cc:385] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:27:44.503078  6865 leader_election.cc:343] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.503711  7102 leader_election.cc:290] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:44.502730  7096 raft_consensus.cc:740] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.505540  7096 consensus_queue.cc:260] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.509438  7096 ts_tablet_manager.cc:1434] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:27:44.506150  6816 tablet_service.cc:1511] Processing CreateTablet for tablet f932aaa29a8443d4b251dea7c9fc0f54 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:27:44.515508  6816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f932aaa29a8443d4b251dea7c9fc0f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.497946  7094 ts_tablet_manager.cc:1403] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.003s
I20260709 12:27:44.515699  6961 tablet_service.cc:1511] Processing CreateTablet for tablet 5df4f6356b53461e89321a089ab9b65f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:27:44.516045  6889 tablet_service.cc:1511] Processing CreateTablet for tablet 5df4f6356b53461e89321a089ab9b65f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:27:44.517047  6961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df4f6356b53461e89321a089ab9b65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.517303  6889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df4f6356b53461e89321a089ab9b65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.518970  7094 raft_consensus.cc:359] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.519924  7096 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:44.519680  6891 tablet_service.cc:1511] Processing CreateTablet for tablet f932aaa29a8443d4b251dea7c9fc0f54 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:27:44.520978  6891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f932aaa29a8443d4b251dea7c9fc0f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.519688  7094 raft_consensus.cc:385] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.527666  6809 tablet_service.cc:1511] Processing CreateTablet for tablet 07ecb64e39814c7a961ebb058d596cbc (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:27:44.528528  7036 tablet_service.cc:1511] Processing CreateTablet for tablet 07ecb64e39814c7a961ebb058d596cbc (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:27:44.529004  6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ecb64e39814c7a961ebb058d596cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.529304  6812 tablet_service.cc:1511] Processing CreateTablet for tablet a690e13ef48d439cb389b897c6b9fcc6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:27:44.529773  7036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ecb64e39814c7a961ebb058d596cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.530570  6812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a690e13ef48d439cb389b897c6b9fcc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.531210  7038 tablet_service.cc:1511] Processing CreateTablet for tablet 5df4f6356b53461e89321a089ab9b65f (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:27:44.532413  7038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5df4f6356b53461e89321a089ab9b65f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.515740  6963 tablet_service.cc:1511] Processing CreateTablet for tablet 07ecb64e39814c7a961ebb058d596cbc (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:27:44.535960  6963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ecb64e39814c7a961ebb058d596cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.536510  7095 tablet_bootstrap.cc:492] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:44.537072  7095 ts_tablet_manager.cc:1403] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.174s	user 0.017s	sys 0.000s
I20260709 12:27:44.538296  7094 raft_consensus.cc:740] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.550957  7094 consensus_queue.cc:260] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.553242  7094 ts_tablet_manager.cc:1434] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.037s	user 0.001s	sys 0.005s
I20260709 12:27:44.554970  7094 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:44.560549  7096 tablet_bootstrap.cc:654] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.565733  6883 tablet_service.cc:1511] Processing CreateTablet for tablet a690e13ef48d439cb389b897c6b9fcc6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:27:44.567129  6883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a690e13ef48d439cb389b897c6b9fcc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.567606  7094 tablet_bootstrap.cc:654] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.553483  7095 raft_consensus.cc:359] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.573540  7095 raft_consensus.cc:385] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.573854  7095 raft_consensus.cc:740] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.574659  7095 consensus_queue.cc:260] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.575521  6961 tablet_service.cc:1511] Processing CreateTablet for tablet a690e13ef48d439cb389b897c6b9fcc6 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:27:44.580456  7097 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:44.582947  7097 ts_tablet_manager.cc:1403] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.147s	user 0.009s	sys 0.004s
I20260709 12:27:44.583212  6961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a690e13ef48d439cb389b897c6b9fcc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.584221  7035 tablet_service.cc:1511] Processing CreateTablet for tablet 3b065e90800f40379b501a6238c6fdcd (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:27:44.585517  7035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b065e90800f40379b501a6238c6fdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.586144  7097 raft_consensus.cc:359] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.586913  7097 raft_consensus.cc:385] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.596832  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b21f93c356074f488a5be203ff0b1988" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:44.597563  7050 raft_consensus.cc:2468] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:44.600314  6816 tablet_service.cc:1511] Processing CreateTablet for tablet 3b065e90800f40379b501a6238c6fdcd (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:27:44.598764  6889 tablet_service.cc:1511] Processing CreateTablet for tablet 3b065e90800f40379b501a6238c6fdcd (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:27:44.600819  7038 tablet_service.cc:1511] Processing CreateTablet for tablet 4e2b62ad0158424696dcdf73faa4f675 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:27:44.601634  6816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b065e90800f40379b501a6238c6fdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.601923  6889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b065e90800f40379b501a6238c6fdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.602027  7038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2b62ad0158424696dcdf73faa4f675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.604404  6866 leader_election.cc:304] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:44.605290  7102 raft_consensus.cc:2804] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:44.605791  7102 raft_consensus.cc:493] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.606129  7102 raft_consensus.cc:3060] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.607823  6809 tablet_service.cc:1511] Processing CreateTablet for tablet 4e2b62ad0158424696dcdf73faa4f675 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:27:44.609078  6809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2b62ad0158424696dcdf73faa4f675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.613449  7102 raft_consensus.cc:515] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.616235  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b21f93c356074f488a5be203ff0b1988" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
W20260709 12:27:44.617636  6865 leader_election.cc:343] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.623720  7095 ts_tablet_manager.cc:1434] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
I20260709 12:27:44.627875  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b21f93c356074f488a5be203ff0b1988" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:44.628494  7050 raft_consensus.cc:3060] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.628741  7095 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:44.633232  6963 tablet_service.cc:1511] Processing CreateTablet for tablet 4e2b62ad0158424696dcdf73faa4f675 (DEFAULT_TABLE table=test-workload [id=c3606615e16145fa8bcb12a725cb8db0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:27:44.634497  6963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2b62ad0158424696dcdf73faa4f675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:44.634809  7102 leader_election.cc:290] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:44.636276  7094 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:44.636047  7050 raft_consensus.cc:2468] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:44.637133  7094 ts_tablet_manager.cc:1403] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.006s
I20260709 12:27:44.637894  6866 leader_election.cc:304] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:44.639107  7102 raft_consensus.cc:2804] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:44.653795  7108 raft_consensus.cc:493] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.654400  7108 raft_consensus.cc:515] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.657255  7108 leader_election.cc:290] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:44.657802  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e962d48b16794d78ad1a367540885945" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:44.588301  7097 raft_consensus.cc:740] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
W20260709 12:27:44.659880  6939 leader_election.cc:343] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.665342  7095 tablet_bootstrap.cc:654] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.669661  7103 raft_consensus.cc:493] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.656118  7094 raft_consensus.cc:359] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.670771  7103 raft_consensus.cc:515] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.674605  7096 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:44.675439  7094 raft_consensus.cc:385] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.716694  7096 ts_tablet_manager.cc:1403] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.202s	user 0.052s	sys 0.036s
I20260709 12:27:44.681684  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea38fed35e6549b3917d485f9af88382" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:44.719038  7013 leader_election.cc:343] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.681164  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea38fed35e6549b3917d485f9af88382" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:44.721755  7016 leader_election.cc:343] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.721673  7096 raft_consensus.cc:359] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.722656  7016 leader_election.cc:304] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:44.722951  7096 raft_consensus.cc:385] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.724980  7094 raft_consensus.cc:740] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.725049  7102 raft_consensus.cc:697] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:44.723651  7096 raft_consensus.cc:740] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.676470  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e962d48b16794d78ad1a367540885945" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:44.678570  7103 leader_election.cc:290] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:44.728516  7094 consensus_queue.cc:260] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.728766  7102 consensus_queue.cc:237] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.730772  7094 ts_tablet_manager.cc:1434] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.002s
I20260709 12:27:44.715758  7097 consensus_queue.cc:260] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.731989  7096 consensus_queue.cc:260] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.729374  7103 raft_consensus.cc:2749] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:44.740079  7140 raft_consensus.cc:493] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.745348  7050 raft_consensus.cc:2468] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:44.745489  7140 raft_consensus.cc:515] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.747845  7140 leader_election.cc:290] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:44.748155  6940 leader_election.cc:304] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:44.741111  7095 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:44.750095  7095 ts_tablet_manager.cc:1403] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.122s	user 0.029s	sys 0.042s
I20260709 12:27:44.749647  6725 catalog_manager.cc:5697] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:44.750691  7108 raft_consensus.cc:2804] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:44.739049  7097 ts_tablet_manager.cc:1434] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.156s	user 0.006s	sys 0.006s
I20260709 12:27:44.751075  7108 raft_consensus.cc:493] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.751468  7108 raft_consensus.cc:3060] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.752033  7097 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:44.749678  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:44.754462  7095 raft_consensus.cc:359] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.755177  7095 raft_consensus.cc:385] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:27:44.755376  7013 leader_election.cc:343] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.755461  7095 raft_consensus.cc:740] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.750034  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:44.756573  7095 consensus_queue.cc:260] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:27:44.757263  7013 leader_election.cc:343] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.757740  7013 leader_election.cc:304] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:44.758842  7095 ts_tablet_manager.cc:1434] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:27:44.758842  7140 raft_consensus.cc:2749] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:44.759557  7097 tablet_bootstrap.cc:654] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.760006  7095 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:44.767635  7095 tablet_bootstrap.cc:654] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.770458  7096 ts_tablet_manager.cc:1434] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 12:27:44.771878  7094 tablet_bootstrap.cc:492] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:44.773542  7096 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:44.787364  7095 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:44.787979  7095 ts_tablet_manager.cc:1403] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.028s	user 0.024s	sys 0.003s
I20260709 12:27:44.792145  7096 tablet_bootstrap.cc:654] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.796641  7108 raft_consensus.cc:515] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.797514  7095 raft_consensus.cc:359] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.804544  7095 raft_consensus.cc:385] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.804913  7095 raft_consensus.cc:740] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.805836  7095 consensus_queue.cc:260] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.800504  7094 tablet_bootstrap.cc:654] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.808501  7095 ts_tablet_manager.cc:1434] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:27:44.809619  7095 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:44.811018  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e962d48b16794d78ad1a367540885945" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:44.811604  7111 raft_consensus.cc:493] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.812146  7111 raft_consensus.cc:515] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:27:44.813099  6939 leader_election.cc:343] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.815770  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e962d48b16794d78ad1a367540885945" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:44.816556  7050 raft_consensus.cc:3060] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.810969  7108 leader_election.cc:290] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:44.823664  7095 tablet_bootstrap.cc:654] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.814669  7111 leader_election.cc:290] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:44.826915  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2693326c6b49989473c3de8c70d40a" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:44.828504  6793 leader_election.cc:343] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.830899  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2693326c6b49989473c3de8c70d40a" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:44.832516  7050 raft_consensus.cc:2468] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
W20260709 12:27:44.833549  6791 leader_election.cc:343] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.834054  6791 leader_election.cc:304] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:44.836115  6940 leader_election.cc:304] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:44.837492  7111 raft_consensus.cc:2749] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:44.838886  7108 raft_consensus.cc:2804] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:44.839296  7097 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:44.839489  7096 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:44.839841  7097 ts_tablet_manager.cc:1403] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.088s	user 0.031s	sys 0.040s
I20260709 12:27:44.840215  7096 ts_tablet_manager.cc:1403] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.067s	user 0.017s	sys 0.004s
I20260709 12:27:44.840384  7108 raft_consensus.cc:697] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:44.841478  7108 consensus_queue.cc:237] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.843112  7097 raft_consensus.cc:359] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.843859  7097 raft_consensus.cc:385] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.844157  7097 raft_consensus.cc:740] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.845512  7097 consensus_queue.cc:260] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.848441  7097 ts_tablet_manager.cc:1434] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:27:44.850160  7094 tablet_bootstrap.cc:492] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:44.850905  7094 ts_tablet_manager.cc:1403] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.079s	user 0.012s	sys 0.012s
I20260709 12:27:44.851354  7097 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:44.859263  7094 raft_consensus.cc:359] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.861035  7097 tablet_bootstrap.cc:654] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.844358  7096 raft_consensus.cc:359] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.862179  7094 raft_consensus.cc:385] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.898701  7094 raft_consensus.cc:740] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.900784  7094 consensus_queue.cc:260] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.905704  7097 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:44.906541  7094 ts_tablet_manager.cc:1434] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.003s
I20260709 12:27:44.907017  7097 ts_tablet_manager.cc:1403] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.008s
I20260709 12:27:44.909041  7094 tablet_bootstrap.cc:492] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:44.908764  6725 catalog_manager.cc:5697] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:44.913224  7097 raft_consensus.cc:359] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.865036  7095 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:44.915247  7095 ts_tablet_manager.cc:1403] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.106s	user 0.023s	sys 0.016s
I20260709 12:27:44.914171  7097 raft_consensus.cc:385] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.919344  7097 raft_consensus.cc:740] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.919514  7094 tablet_bootstrap.cc:654] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.920725  7097 consensus_queue.cc:260] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.918716  7096 raft_consensus.cc:385] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.923893  7096 raft_consensus.cc:740] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.924966  7097 ts_tablet_manager.cc:1434] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.005s
I20260709 12:27:44.925588  7096 consensus_queue.cc:260] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.927371  7095 raft_consensus.cc:359] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.928519  7095 raft_consensus.cc:385] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.928960  7095 raft_consensus.cc:740] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.930039  7096 ts_tablet_manager.cc:1434] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.089s	user 0.012s	sys 0.000s
I20260709 12:27:44.930320  7095 consensus_queue.cc:260] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.932121  7111 raft_consensus.cc:493] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.932531  7096 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:44.932938  7111 raft_consensus.cc:515] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.932188  7108 raft_consensus.cc:493] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.934360  7108 raft_consensus.cc:515] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.937041  7111 leader_election.cc:290] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:44.940075  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:44.940570  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:44.941116  7050 raft_consensus.cc:2468] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:44.941676  7096 tablet_bootstrap.cc:654] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.941519  6902 raft_consensus.cc:2468] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:44.943460  6793 leader_election.cc:304] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:44.944476  7111 raft_consensus.cc:2804] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:44.944636  7094 tablet_bootstrap.cc:492] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:44.944821  7111 raft_consensus.cc:493] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.945019  7108 leader_election.cc:290] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:44.945248  7094 ts_tablet_manager.cc:1403] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.005s
I20260709 12:27:44.945271  7111 raft_consensus.cc:3060] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.946864  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea38fed35e6549b3917d485f9af88382" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:44.948037  7050 raft_consensus.cc:2468] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:44.949195  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea38fed35e6549b3917d485f9af88382" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:44.949362  6940 leader_election.cc:304] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:44.949903  7094 raft_consensus.cc:359] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.950558  7110 raft_consensus.cc:493] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:44.951007  6942 leader_election.cc:343] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.951340  7110 raft_consensus.cc:515] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:44.954104  7095 ts_tablet_manager.cc:1434] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.008s
I20260709 12:27:44.954723  7097 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:44.954835  7110 leader_election.cc:290] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:44.956771  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251b19e52f0146aa9a9cfc985e2763b4" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:44.950683  7094 raft_consensus.cc:385] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:44.950600  7108 raft_consensus.cc:2804] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:27:44.958734  6939 leader_election.cc:343] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.959095  7108 raft_consensus.cc:493] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.959522  7108 raft_consensus.cc:3060] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.963339  7111 raft_consensus.cc:515] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:44.958384  7094 raft_consensus.cc:740] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:44.965765  7095 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:44.972348  7151 raft_consensus.cc:493] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.974154  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:44.975600  6902 raft_consensus.cc:3060] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.975688  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:44.976452  7049 raft_consensus.cc:3060] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.977921  7108 raft_consensus.cc:515] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.979074  7097 tablet_bootstrap.cc:654] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.981281  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea38fed35e6549b3917d485f9af88382" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:44.981776  7108 leader_election.cc:290] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
W20260709 12:27:44.983058  6942 leader_election.cc:343] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.983382  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea38fed35e6549b3917d485f9af88382" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:44.983603  7151 raft_consensus.cc:515] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:44.984071  7048 raft_consensus.cc:3060] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.987504  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed0915d1da4514bc4c51eddfcf5e31" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:44.987540  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed0915d1da4514bc4c51eddfcf5e31" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:44.988656  6901 raft_consensus.cc:2468] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
W20260709 12:27:44.989042  6939 leader_election.cc:343] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:44.990269  6942 leader_election.cc:304] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:44.990784  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251b19e52f0146aa9a9cfc985e2763b4" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:44.992236  7110 raft_consensus.cc:2804] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:44.992668  7110 raft_consensus.cc:493] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.993336  7110 raft_consensus.cc:3060] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:44.973297  7094 consensus_queue.cc:260] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:44.974551  7111 leader_election.cc:290] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.001402  7102 raft_consensus.cc:493] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.003917  7095 tablet_bootstrap.cc:654] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:44.998401  7151 leader_election.cc:290] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
W20260709 12:27:45.005298  6939 consensus_peers.cc:597] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:44.992630  6940 leader_election.cc:343] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.011647  6940 leader_election.cc:304] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:45.011524  7102 raft_consensus.cc:515] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.014238  7102 leader_election.cc:290] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:45.014626  7096 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:45.015195  7096 ts_tablet_manager.cc:1403] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.083s	user 0.025s	sys 0.007s
I20260709 12:27:45.015148  7048 raft_consensus.cc:2468] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:45.016072  7108 consensus_queue.cc:1048] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:45.020078  7151 raft_consensus.cc:2749] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:45.016628  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "052922f90cab431dba48fb4991dee3fb" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:45.021838  7049 raft_consensus.cc:2468] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:45.019416  7096 raft_consensus.cc:359] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.022890  7096 raft_consensus.cc:385] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.023288  7096 raft_consensus.cc:740] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.023527  6791 leader_election.cc:304] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:45.024385  6902 raft_consensus.cc:2468] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:45.026170  6940 leader_election.cc:304] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
W20260709 12:27:45.026985  6866 leader_election.cc:343] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.027693  7108 raft_consensus.cc:2804] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.028288  7108 raft_consensus.cc:697] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:45.029207  7108 consensus_queue.cc:237] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.032953  7097 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:45.033488  7097 ts_tablet_manager.cc:1403] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.079s	user 0.011s	sys 0.011s
I20260709 12:27:45.020274  7111 raft_consensus.cc:493] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.040295  7157 raft_consensus.cc:2804] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.042626  7111 raft_consensus.cc:515] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.044782  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "052922f90cab431dba48fb4991dee3fb" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:45.024366  7096 consensus_queue.cc:260] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:27:45.047358  6865 leader_election.cc:343] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.048106  6865 leader_election.cc:304] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:45.049342  7096 ts_tablet_manager.cc:1434] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:27:45.049767  7102 raft_consensus.cc:2749] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:45.052167  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.053054  6901 raft_consensus.cc:2468] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:45.054513  6793 leader_election.cc:304] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:45.041762  7157 raft_consensus.cc:697] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:45.045701  7111 leader_election.cc:290] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:45.063606  7097 raft_consensus.cc:359] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.064350  7097 raft_consensus.cc:385] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.064657  7097 raft_consensus.cc:740] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.065729  7096 tablet_bootstrap.cc:492] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:45.065495  7097 consensus_queue.cc:260] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.066774  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:45.067876  7097 ts_tablet_manager.cc:1434] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.002s
W20260709 12:27:45.069788  6790 leader_election.cc:343] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.069159  7097 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:45.075697  7110 raft_consensus.cc:515] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.077728  7097 tablet_bootstrap.cc:654] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.082327  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed0915d1da4514bc4c51eddfcf5e31" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
W20260709 12:27:45.086825  6939 leader_election.cc:343] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.078474  7111 raft_consensus.cc:2804] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:45.089561  7111 raft_consensus.cc:493] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.091287  7111 raft_consensus.cc:3060] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.092056  6722 catalog_manager.cc:5697] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:45.094462  7157 consensus_queue.cc:237] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.097084  7110 leader_election.cc:290] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.101658  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ed0915d1da4514bc4c51eddfcf5e31" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:45.102548  6901 raft_consensus.cc:3060] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.101692  7096 tablet_bootstrap.cc:654] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.107919  6723 catalog_manager.cc:5697] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:45.109078  7159 raft_consensus.cc:493] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.109865  7159 raft_consensus.cc:515] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.113049  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:45.113268  6976 raft_consensus.cc:3060] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.113709  7049 raft_consensus.cc:2468] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:45.115535  6791 leader_election.cc:304] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:45.118033  7157 raft_consensus.cc:2804] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:45.119611  7102 consensus_queue.cc:1048] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:45.120465  7157 raft_consensus.cc:493] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.121145  7157 raft_consensus.cc:3060] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.123862  7095 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:45.125226  6901 raft_consensus.cc:2468] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:45.125777  7095 ts_tablet_manager.cc:1403] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.160s	user 0.018s	sys 0.035s
I20260709 12:27:45.128696  7094 ts_tablet_manager.cc:1434] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.183s	user 0.009s	sys 0.011s
I20260709 12:27:45.129997  7094 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:45.130903  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.131265  7095 raft_consensus.cc:359] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.132104  7159 leader_election.cc:290] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.132138  7095 raft_consensus.cc:385] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.133025  7095 raft_consensus.cc:740] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.133836  7095 consensus_queue.cc:260] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.134197  7102 consensus_queue.cc:1048] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:45.134830  6902 raft_consensus.cc:2468] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:45.135747  6942 leader_election.cc:304] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:45.137452  7108 raft_consensus.cc:2804] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.138419  7108 raft_consensus.cc:697] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:45.144508  7095 ts_tablet_manager.cc:1434] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:27:45.142170  7111 raft_consensus.cc:515] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.142117  7108 consensus_queue.cc:237] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.175051  7111 leader_election.cc:290] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:45.176584  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:45.178570  7094 tablet_bootstrap.cc:654] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.179098  7095 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
W20260709 12:27:45.179260  6790 leader_election.cc:343] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.177560  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:45.180470  6901 raft_consensus.cc:3060] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.187707  6901 raft_consensus.cc:2468] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:45.149641  7096 tablet_bootstrap.cc:492] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:45.189366  7096 ts_tablet_manager.cc:1403] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.124s	user 0.012s	sys 0.010s
I20260709 12:27:45.189252  6793 leader_election.cc:304] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:45.192579  7096 raft_consensus.cc:359] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.147671  7097 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:45.193260  7096 raft_consensus.cc:385] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.193583  7096 raft_consensus.cc:740] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.174134  7157 raft_consensus.cc:515] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.194700  7111 raft_consensus.cc:2804] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.194510  7096 consensus_queue.cc:260] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.196017  7111 raft_consensus.cc:697] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:45.199226  7095 tablet_bootstrap.cc:654] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.199395  7111 consensus_queue.cc:237] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.203480  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:45.204129  7048 raft_consensus.cc:3060] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.203655  6722 catalog_manager.cc:5697] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:45.211524  7048 raft_consensus.cc:2468] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:45.212690  6791 leader_election.cc:304] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:45.213840  7164 raft_consensus.cc:2804] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.214458  7164 raft_consensus.cc:697] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:45.214779  7096 ts_tablet_manager.cc:1434] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:27:45.215448  7164 consensus_queue.cc:237] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.220382  7096 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:45.226879  7094 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:45.227487  7094 ts_tablet_manager.cc:1403] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.098s	user 0.018s	sys 0.022s
I20260709 12:27:45.229442  7096 tablet_bootstrap.cc:654] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.231063  7094 raft_consensus.cc:359] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.231876  7094 raft_consensus.cc:385] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.232255  7094 raft_consensus.cc:740] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.233431  7094 consensus_queue.cc:260] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
W20260709 12:27:45.234584  6939 consensus_peers.cc:597] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:45.236960  7094 ts_tablet_manager.cc:1434] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:27:45.238631  7094 tablet_bootstrap.cc:492] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:45.248800  7094 tablet_bootstrap.cc:654] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.196220  7157 leader_election.cc:290] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.197372  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:45.252422  6901 raft_consensus.cc:3060] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.198143  7097 ts_tablet_manager.cc:1403] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.129s	user 0.009s	sys 0.013s
I20260709 12:27:45.262460  6901 raft_consensus.cc:2468] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:45.262212  7097 raft_consensus.cc:359] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.263283  7097 raft_consensus.cc:385] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.263659  7097 raft_consensus.cc:740] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.264849  7097 consensus_queue.cc:260] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.266057  6722 catalog_manager.cc:5697] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:45.270689  7094 tablet_bootstrap.cc:492] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:45.271389  7094 ts_tablet_manager.cc:1403] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.006s
I20260709 12:27:45.276008  7094 raft_consensus.cc:359] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.280145  7094 raft_consensus.cc:385] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.280623  7094 raft_consensus.cc:740] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.276932  7097 ts_tablet_manager.cc:1434] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:27:45.282743  7097 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:45.283195  7157 raft_consensus.cc:493] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.286580  7157 raft_consensus.cc:515] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.290400  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:45.292325  6790 leader_election.cc:343] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.293394  6723 catalog_manager.cc:5697] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:45.293488  7094 consensus_queue.cc:260] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.303615  7157 leader_election.cc:290] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:45.304270  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:45.301864  7097 tablet_bootstrap.cc:654] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.306273  7048 raft_consensus.cc:2468] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:45.305495  7094 ts_tablet_manager.cc:1434] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.003s
I20260709 12:27:45.307421  6791 leader_election.cc:304] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:45.308290  7094 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:45.308636  7157 raft_consensus.cc:2804] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:45.309022  7157 raft_consensus.cc:493] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.309356  7157 raft_consensus.cc:3060] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.315186  7094 tablet_bootstrap.cc:654] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.319603  7157 raft_consensus.cc:515] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.323419  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
W20260709 12:27:45.325228  6790 leader_election.cc:343] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.328214  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:45.328887  7048 raft_consensus.cc:3060] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.330114  7094 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:45.330647  7094 ts_tablet_manager.cc:1403] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.001s
I20260709 12:27:45.333526  7094 raft_consensus.cc:359] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.334156  7094 raft_consensus.cc:385] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.334450  7094 raft_consensus.cc:740] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.335167  7094 consensus_queue.cc:260] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.337090  7048 raft_consensus.cc:2468] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:45.338647  6791 leader_election.cc:304] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:45.339974  7111 raft_consensus.cc:2804] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.340664  7111 raft_consensus.cc:697] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:45.341807  7111 consensus_queue.cc:237] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.337277  7094 ts_tablet_manager.cc:1434] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:27:45.345386  7108 raft_consensus.cc:493] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.345651  7094 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:45.345997  7108 raft_consensus.cc:515] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.346442  7095 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:45.347074  7095 ts_tablet_manager.cc:1403] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.173s	user 0.048s	sys 0.084s
I20260709 12:27:45.349035  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:45.349668  7108 leader_election.cc:290] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.351541  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.350625  6827 raft_consensus.cc:2468] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:45.352918  7094 tablet_bootstrap.cc:654] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:45.353156  6942 leader_election.cc:343] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.353780  6939 leader_election.cc:304] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:45.354764  7108 raft_consensus.cc:2804] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:45.353770  7164 raft_consensus.cc:493] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.355376  7108 raft_consensus.cc:493] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.355862  7108 raft_consensus.cc:3060] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.355522  7164 raft_consensus.cc:515] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.357019  7096 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:45.357506  7096 ts_tablet_manager.cc:1403] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.137s	user 0.036s	sys 0.064s
I20260709 12:27:45.357729  7140 raft_consensus.cc:493] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.358058  7164 leader_election.cc:290] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:45.358626  7097 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:45.359102  7097 ts_tablet_manager.cc:1403] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.000s
I20260709 12:27:45.359687  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e962d48b16794d78ad1a367540885945" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:45.360704  7049 raft_consensus.cc:2393] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:45.360404  7096 raft_consensus.cc:359] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.361117  7096 raft_consensus.cc:385] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.358235  7140 raft_consensus.cc:515] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.364315  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.361418  7096 raft_consensus.cc:740] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.365768  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e962d48b16794d78ad1a367540885945" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:45.366083  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:45.366204  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:45.349989  7095 raft_consensus.cc:359] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.367307  7095 raft_consensus.cc:385] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.367825  7095 raft_consensus.cc:740] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
W20260709 12:27:45.367974  7016 leader_election.cc:343] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.365934  7096 consensus_queue.cc:260] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.368579  7095 consensus_queue.cc:260] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.368749  6790 leader_election.cc:304] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
W20260709 12:27:45.370034  7013 leader_election.cc:343] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.370509  7013 leader_election.cc:304] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:45.371167  7095 ts_tablet_manager.cc:1434] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:27:45.380945  7095 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:45.380220  6723 catalog_manager.cc:5697] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:45.372360  7140 leader_election.cc:290] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:45.382014  7140 raft_consensus.cc:2749] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:45.389187  7095 tablet_bootstrap.cc:654] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.376183  7157 leader_election.cc:290] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:45.376179  7110 consensus_queue.cc:1048] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:45.378724  7097 raft_consensus.cc:359] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.398308  7097 raft_consensus.cc:385] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.398685  7097 raft_consensus.cc:740] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.399624  7097 consensus_queue.cc:260] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.401285  7181 raft_consensus.cc:493] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.401866  7097 ts_tablet_manager.cc:1434] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 12:27:45.401949  7096 ts_tablet_manager.cc:1434] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.044s	user 0.022s	sys 0.015s
I20260709 12:27:45.371564  7164 raft_consensus.cc:3060] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.404244  7096 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:45.407799  7140 raft_consensus.cc:493] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.408303  7140 raft_consensus.cc:515] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.410513  7140 leader_election.cc:290] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.412070  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.413030  7097 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
W20260709 12:27:45.414067  7016 leader_election.cc:343] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.418121  7096 tablet_bootstrap.cc:654] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.401782  7181 raft_consensus.cc:515] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.431831  7164 raft_consensus.cc:2749] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:45.436959  7181 leader_election.cc:290] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.437204  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2693326c6b49989473c3de8c70d40a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.437981  6901 raft_consensus.cc:2468] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:45.439640  7016 leader_election.cc:304] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:45.440927  7140 raft_consensus.cc:2804] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:45.441310  7140 raft_consensus.cc:493] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.441660  7140 raft_consensus.cc:3060] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:45.442436  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 72546.2us
I20260709 12:27:45.426093  7097 tablet_bootstrap.cc:654] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.433769  7096 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:45.435022  7095 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:45.445583  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2693326c6b49989473c3de8c70d40a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:45.446126  7095 ts_tablet_manager.cc:1403] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.066s	user 0.036s	sys 0.017s
I20260709 12:27:45.447635  6826 raft_consensus.cc:2468] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:45.449096  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:45.449258  7095 raft_consensus.cc:359] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.449898  7095 raft_consensus.cc:385] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.446076  7096 ts_tablet_manager.cc:1403] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.042s	user 0.021s	sys 0.000s
I20260709 12:27:45.450184  7095 raft_consensus.cc:740] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.451045  7095 consensus_queue.cc:260] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
W20260709 12:27:45.451627  7013 leader_election.cc:343] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.453895  7096 raft_consensus.cc:359] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.454396  7013 leader_election.cc:304] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:45.454802  7096 raft_consensus.cc:385] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.455237  7096 raft_consensus.cc:740] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.455941  7182 raft_consensus.cc:2749] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:45.455991  7096 consensus_queue.cc:260] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.456825  7095 ts_tablet_manager.cc:1434] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:27:45.457969  7095 tablet_bootstrap.cc:492] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:45.460259  7108 raft_consensus.cc:515] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.463078  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:45.463634  6827 raft_consensus.cc:3060] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.463965  7095 tablet_bootstrap.cc:654] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.464594  7140 raft_consensus.cc:515] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.467229  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:45.467888  6901 raft_consensus.cc:3060] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.469118  7094 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:45.469676  7094 ts_tablet_manager.cc:1403] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.124s	user 0.018s	sys 0.014s
I20260709 12:27:45.472396  7094 raft_consensus.cc:359] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.473023  7094 raft_consensus.cc:385] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.473277  7094 raft_consensus.cc:740] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.473963  7094 consensus_queue.cc:260] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.476055  7094 ts_tablet_manager.cc:1434] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:27:45.476281  6901 raft_consensus.cc:2468] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:45.477092  7094 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:45.477458  6942 leader_election.cc:304] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:45.478435  7096 ts_tablet_manager.cc:1434] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 12:27:45.479275  7110 raft_consensus.cc:2804] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.479558  7096 tablet_bootstrap.cc:492] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:45.479815  7110 raft_consensus.cc:697] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:45.480767  7110 consensus_queue.cc:237] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.483624  7164 consensus_queue.cc:1048] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:45.486542  7094 tablet_bootstrap.cc:654] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.486665  7108 leader_election.cc:290] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.489920  7140 leader_election.cc:290] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.489699  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2693326c6b49989473c3de8c70d40a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:45.489619  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2693326c6b49989473c3de8c70d40a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:45.490625  6826 raft_consensus.cc:3060] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.491108  6901 raft_consensus.cc:3060] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.492605  7096 tablet_bootstrap.cc:654] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.503466  6901 raft_consensus.cc:2468] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:45.505095  7016 leader_election.cc:304] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:45.506291  7140 raft_consensus.cc:2804] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.506914  7140 raft_consensus.cc:697] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:45.508009  7140 consensus_queue.cc:237] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.540174  7097 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:45.540797  7097 ts_tablet_manager.cc:1403] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.128s	user 0.012s	sys 0.008s
I20260709 12:27:45.544462  7097 raft_consensus.cc:359] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.545385  7097 raft_consensus.cc:385] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.545763  7097 raft_consensus.cc:740] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.546916  7097 consensus_queue.cc:260] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.560356  7108 consensus_queue.cc:1048] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:45.563565  6827 raft_consensus.cc:2468] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:45.571355  7097 ts_tablet_manager.cc:1434] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 12:27:45.583094  6825 raft_consensus.cc:3060] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.588599  7108 consensus_queue.cc:1048] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:45.595438  6723 catalog_manager.cc:5697] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:45.597654  7164 raft_consensus.cc:493] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.612569  7164 raft_consensus.cc:515] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.627719  6976 raft_consensus.cc:3060] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.631547  7111 consensus_queue.cc:1048] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:45.637692  7108 raft_consensus.cc:493] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.638250  7108 raft_consensus.cc:515] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.639602  7094 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:45.640204  7094 ts_tablet_manager.cc:1403] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.163s	user 0.020s	sys 0.056s
I20260709 12:27:45.640461  7108 leader_election.cc:290] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:45.642791  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "052922f90cab431dba48fb4991dee3fb" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.643668  6900 raft_consensus.cc:2468] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:45.643858  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "052922f90cab431dba48fb4991dee3fb" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:45.644937  6942 leader_election.cc:304] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:45.645787  7108 raft_consensus.cc:2804] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:45.646278  7095 tablet_bootstrap.cc:492] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:45.646185  7108 raft_consensus.cc:493] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.646874  7095 ts_tablet_manager.cc:1403] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.189s	user 0.043s	sys 0.043s
I20260709 12:27:45.646842  7108 raft_consensus.cc:3060] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.648198  7157 consensus_queue.cc:1048] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:45.649538  7157 consensus_queue.cc:1048] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:27:45.650857  6940 leader_election.cc:343] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.651757  7164 leader_election.cc:290] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.652413  6827 raft_consensus.cc:3060] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.652875  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847f2ce68f10434cb350d63a6573b3bd" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:45.653010  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "847f2ce68f10434cb350d63a6573b3bd" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.658852  6793 leader_election.cc:304] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:45.668146  7097 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:45.533957  6826 raft_consensus.cc:2468] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:45.680018  7154 consensus_queue.cc:1048] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:27:45.687170  6942 consensus_peers.cc:597] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:45.689329  7096 tablet_bootstrap.cc:492] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:45.689872  7096 ts_tablet_manager.cc:1403] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.211s	user 0.048s	sys 0.067s
I20260709 12:27:45.693387  7095 raft_consensus.cc:359] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.694142  7095 raft_consensus.cc:385] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.694509  7095 raft_consensus.cc:740] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.695379  7095 consensus_queue.cc:260] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.708348  7097 tablet_bootstrap.cc:654] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.732295  7094 raft_consensus.cc:359] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.733368  7094 raft_consensus.cc:385] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.737257  6974 raft_consensus.cc:3060] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.733718  7094 raft_consensus.cc:740] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.743803  7094 consensus_queue.cc:260] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.699115  7108 raft_consensus.cc:515] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
W20260709 12:27:45.747862  7188 log.cc:927] Time spent T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:27:45.750468  7138 raft_consensus.cc:493] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.750989  7138 raft_consensus.cc:515] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.753194  7138 leader_election.cc:290] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:45.754063  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea38fed35e6549b3917d485f9af88382" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:45.755942  7096 raft_consensus.cc:359] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.756631  7096 raft_consensus.cc:385] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.756906  7096 raft_consensus.cc:740] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.756839  6723 catalog_manager.cc:5697] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:45.757759  7096 consensus_queue.cc:260] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.759168  7095 ts_tablet_manager.cc:1434] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.112s	user 0.008s	sys 0.001s
W20260709 12:27:45.760077  7197 log.cc:927] Time spent T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:27:45.760233  7108 leader_election.cc:290] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:45.760479  7095 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:45.761850  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "052922f90cab431dba48fb4991dee3fb" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:45.762300  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "052922f90cab431dba48fb4991dee3fb" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:45.762490  6901 raft_consensus.cc:3060] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.762913  7050 raft_consensus.cc:3060] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.771932  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea38fed35e6549b3917d485f9af88382" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:45.777665  6866 leader_election.cc:304] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:45.779455  7138 raft_consensus.cc:2749] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:45.781080  7096 ts_tablet_manager.cc:1434] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.003s
I20260709 12:27:45.781283  7157 raft_consensus.cc:2749] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:45.782249  7096 tablet_bootstrap.cc:492] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:45.786005  7097 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:45.786490  7097 ts_tablet_manager.cc:1403] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.119s	user 0.003s	sys 0.023s
I20260709 12:27:45.787781  7094 ts_tablet_manager.cc:1434] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.147s	user 0.008s	sys 0.002s
I20260709 12:27:45.792256  7097 raft_consensus.cc:359] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.792939  7097 raft_consensus.cc:385] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.795665  7095 tablet_bootstrap.cc:654] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.793833  7097 raft_consensus.cc:740] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.794857  7181 raft_consensus.cc:493] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.796598  7181 raft_consensus.cc:515] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.798990  7181 leader_election.cc:290] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:45.796598  7097 consensus_queue.cc:260] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.807595  7164 consensus_queue.cc:1048] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:45.808794  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251b19e52f0146aa9a9cfc985e2763b4" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:45.811066  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251b19e52f0146aa9a9cfc985e2763b4" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:45.815253  7013 leader_election.cc:343] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.821995  7097 ts_tablet_manager.cc:1434] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.002s
I20260709 12:27:45.811317  6976 raft_consensus.cc:2468] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:45.829458  7013 leader_election.cc:304] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:45.830909  7204 consensus_queue.cc:1048] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:45.831929  7181 raft_consensus.cc:2804] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:45.832479  7181 raft_consensus.cc:493] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.832909  7181 raft_consensus.cc:3060] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.840308  7094 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:45.844228  7097 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:45.843753  6901 raft_consensus.cc:2468] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:45.849416  7096 tablet_bootstrap.cc:654] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.846344  7050 raft_consensus.cc:2468] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:45.842067  7159 consensus_queue.cc:1048] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:27:45.875932  7095 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:45.876444  7095 ts_tablet_manager.cc:1403] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.116s	user 0.009s	sys 0.008s
I20260709 12:27:45.889237  7095 raft_consensus.cc:359] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.889953  7095 raft_consensus.cc:385] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.890239  7095 raft_consensus.cc:740] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.890946  7095 consensus_queue.cc:260] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:45.895542  7097 tablet_bootstrap.cc:654] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.872932  6942 leader_election.cc:304] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:45.896337  7108 raft_consensus.cc:2804] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.896879  7108 raft_consensus.cc:697] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:45.897719  7108 consensus_queue.cc:237] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.904525  7095 ts_tablet_manager.cc:1434] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:27:45.905264  7111 consensus_queue.cc:1048] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:45.905937  7095 tablet_bootstrap.cc:492] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:45.902025  7181 raft_consensus.cc:515] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.911753  7181 leader_election.cc:290] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:45.913251  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251b19e52f0146aa9a9cfc985e2763b4" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:45.913864  6974 raft_consensus.cc:3060] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.914803  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "251b19e52f0146aa9a9cfc985e2763b4" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:45.919560  7097 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:45.920117  7097 ts_tablet_manager.cc:1403] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.000s
W20260709 12:27:45.922060  7013 leader_election.cc:343] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.924389  7096 tablet_bootstrap.cc:492] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:45.925096  7096 ts_tablet_manager.cc:1403] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.143s	user 0.029s	sys 0.019s
I20260709 12:27:45.931087  6974 raft_consensus.cc:2468] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:45.932121  7013 leader_election.cc:304] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:45.934175  7094 tablet_bootstrap.cc:654] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.933264  7140 raft_consensus.cc:2804] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:45.941191  7140 raft_consensus.cc:697] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:45.938552  6725 catalog_manager.cc:5697] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:45.942269  7140 consensus_queue.cc:237] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.951009  7159 raft_consensus.cc:493] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.951104  7097 raft_consensus.cc:359] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.951611  7159 raft_consensus.cc:515] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:45.951982  7097 raft_consensus.cc:385] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.952402  7097 raft_consensus.cc:740] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.938488  7096 raft_consensus.cc:359] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.953967  7096 raft_consensus.cc:385] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:45.954344  7096 raft_consensus.cc:740] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:45.953343  7097 consensus_queue.cc:260] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:45.961639  7096 consensus_queue.cc:260] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:45.963491  7095 tablet_bootstrap.cc:654] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:45.966872  7159 leader_election.cc:290] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:45.968335  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:45.972702  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
W20260709 12:27:45.974961  6793 leader_election.cc:343] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:45.978595  7094 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:45.978698  7096 ts_tablet_manager.cc:1434] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 12:27:45.979421  7094 ts_tablet_manager.cc:1403] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.157s	user 0.017s	sys 0.009s
I20260709 12:27:45.980182  7096 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:45.973310  7050 raft_consensus.cc:2468] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:45.982540  6791 leader_election.cc:304] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:45.983749  7111 raft_consensus.cc:2804] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:45.984151  7111 raft_consensus.cc:493] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.984498  7111 raft_consensus.cc:3060] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:45.987744  7097 ts_tablet_manager.cc:1434] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 12:27:45.989427  7097 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:45.989794  6723 catalog_manager.cc:5697] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:45.991940  7154 consensus_queue.cc:1048] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:45.993546  7140 consensus_queue.cc:1048] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:45.983336  7094 raft_consensus.cc:359] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.008029  7094 raft_consensus.cc:385] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.008415  7094 raft_consensus.cc:740] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.002027  7111 raft_consensus.cc:515] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.009483  7095 tablet_bootstrap.cc:492] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.009997  7095 ts_tablet_manager.cc:1403] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.104s	user 0.022s	sys 0.003s
I20260709 12:27:46.011132  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:46.011698  7138 raft_consensus.cc:493] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.011758  7049 raft_consensus.cc:3060] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.012301  7138 raft_consensus.cc:515] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.013082  7095 raft_consensus.cc:359] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.013743  7095 raft_consensus.cc:385] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.014003  7095 raft_consensus.cc:740] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.014813  7095 consensus_queue.cc:260] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.015312  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:46.016773  6865 leader_election.cc:343] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.016714  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:46.018688  6865 leader_election.cc:343] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.019093  6865 leader_election.cc:304] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.015110  7138 leader_election.cc:290] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:46.020578  7138 raft_consensus.cc:2749] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.021126  7108 consensus_queue.cc:1048] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.022410  7049 raft_consensus.cc:2468] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:46.023599  6791 leader_election.cc:304] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:46.024757  7205 raft_consensus.cc:2804] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:46.025262  7205 raft_consensus.cc:697] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:46.026077  7205 consensus_queue.cc:237] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.032617  7096 tablet_bootstrap.cc:654] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.010284  7097 tablet_bootstrap.cc:654] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.044250  7223 consensus_queue.cc:1048] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:46.048905  7094 consensus_queue.cc:260] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.056183  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
W20260709 12:27:46.057919  6793 leader_election.cc:343] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.058882  7111 leader_election.cc:290] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:46.065025  6902 raft_consensus.cc:3060] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.068289  7094 ts_tablet_manager.cc:1434] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.003s
I20260709 12:27:46.071326  7096 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:46.071702  7094 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:46.071890  7096 ts_tablet_manager.cc:1403] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.002s
I20260709 12:27:46.074822  7096 raft_consensus.cc:359] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.075481  7096 raft_consensus.cc:385] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.075855  7096 raft_consensus.cc:740] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.076740  7095 ts_tablet_manager.cc:1434] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.066s	user 0.004s	sys 0.020s
I20260709 12:27:46.077809  7095 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.080780  6723 catalog_manager.cc:5697] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:46.083613  7095 tablet_bootstrap.cc:654] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.084071  7094 tablet_bootstrap.cc:654] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.086594  7096 consensus_queue.cc:260] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.095336  7096 ts_tablet_manager.cc:1434] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 12:27:46.096505  7095 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.100379  7095 ts_tablet_manager.cc:1403] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 12:27:46.100821  7096 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:46.103226  7095 raft_consensus.cc:359] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.103888  7095 raft_consensus.cc:385] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.104156  7095 raft_consensus.cc:740] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.104812  7095 consensus_queue.cc:260] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.097146  7221 raft_consensus.cc:493] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.106550  7096 tablet_bootstrap.cc:654] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.106489  7221 raft_consensus.cc:515] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.109169  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:46.110478  6865 leader_election.cc:343] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.111488  7221 leader_election.cc:290] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.112407  7095 ts_tablet_manager.cc:1434] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:27:46.112986  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.113489  7095 tablet_bootstrap.cc:492] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.113875  7049 raft_consensus.cc:2468] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:46.117856  6866 leader_election.cc:304] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.118907  7221 raft_consensus.cc:2804] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.119385  7221 raft_consensus.cc:493] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.119871  7095 tablet_bootstrap.cc:654] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.119796  7221 raft_consensus.cc:3060] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.098400  7097 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:46.126329  7097 ts_tablet_manager.cc:1403] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.000s
I20260709 12:27:46.130753  7095 tablet_bootstrap.cc:492] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.131227  7095 ts_tablet_manager.cc:1403] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 12:27:46.132959  7097 raft_consensus.cc:359] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.133634  7097 raft_consensus.cc:385] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.133947  7097 raft_consensus.cc:740] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.133759  7095 raft_consensus.cc:359] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.134482  7095 raft_consensus.cc:385] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.134775  7095 raft_consensus.cc:740] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.134645  7097 consensus_queue.cc:260] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.135532  7095 consensus_queue.cc:260] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.098604  7094 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:46.136651  7094 ts_tablet_manager.cc:1403] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.001s
I20260709 12:27:46.136770  7097 ts_tablet_manager.cc:1434] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:27:46.138049  7097 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:46.139613  7094 raft_consensus.cc:359] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.140240  7094 raft_consensus.cc:385] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.140488  7094 raft_consensus.cc:740] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.141283  7094 consensus_queue.cc:260] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.145414  7095 ts_tablet_manager.cc:1434] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:27:46.147055  7094 ts_tablet_manager.cc:1434] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 12:27:46.148070  7094 tablet_bootstrap.cc:492] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:46.152944  7222 raft_consensus.cc:493] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.153748  7094 tablet_bootstrap.cc:654] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.153474  7222 raft_consensus.cc:515] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.155704  7097 tablet_bootstrap.cc:654] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.155862  7222 leader_election.cc:290] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.156744  7095 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.157687  7111 raft_consensus.cc:493] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.157606  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75c82b1c500f4214aef5d8ed2567f780" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:46.159214  6865 leader_election.cc:343] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.160072  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75c82b1c500f4214aef5d8ed2567f780" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.160754  7049 raft_consensus.cc:2468] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:46.162353  7111 raft_consensus.cc:515] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.163297  7095 tablet_bootstrap.cc:654] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.165158  7096 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:46.165254  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93204a5a0d0742ad9cd21a8a280af392" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:46.165922  7096 ts_tablet_manager.cc:1403] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.065s	user 0.034s	sys 0.010s
W20260709 12:27:46.167112  6790 leader_election.cc:343] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.168262  6866 leader_election.cc:304] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.169292  7222 raft_consensus.cc:2804] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.169691  7222 raft_consensus.cc:493] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.170326  7111 leader_election.cc:290] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.170033  7222 raft_consensus.cc:3060] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.171573  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93204a5a0d0742ad9cd21a8a280af392" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
W20260709 12:27:46.173127  6791 leader_election.cc:343] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.173599  6791 leader_election.cc:304] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.174432  7111 raft_consensus.cc:2749] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.176151  7221 raft_consensus.cc:515] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.179947  7097 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:46.179714  7094 tablet_bootstrap.cc:492] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:46.180449  7097 ts_tablet_manager.cc:1403] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.000s
I20260709 12:27:46.180727  7094 ts_tablet_manager.cc:1403] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.005s
I20260709 12:27:46.186622  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:46.188632  7096 raft_consensus.cc:359] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
W20260709 12:27:46.189457  6865 leader_election.cc:343] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.194128  7097 raft_consensus.cc:359] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.196734  7097 raft_consensus.cc:385] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.197041  7097 raft_consensus.cc:740] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.197355  7095 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.183219  7222 raft_consensus.cc:515] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.194397  7094 raft_consensus.cc:359] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.197860  7094 raft_consensus.cc:385] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.198356  7094 raft_consensus.cc:740] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.199072  7094 consensus_queue.cc:260] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.199712  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:46.197845  7095 ts_tablet_manager.cc:1403] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.041s	user 0.025s	sys 0.003s
I20260709 12:27:46.201627  7094 ts_tablet_manager.cc:1434] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:27:46.202085  7222 leader_election.cc:290] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.202678  7096 raft_consensus.cc:385] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.203003  7094 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:46.200387  7049 raft_consensus.cc:3060] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.204435  7095 raft_consensus.cc:359] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.205090  7095 raft_consensus.cc:385] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.205371  7095 raft_consensus.cc:740] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.206064  7095 consensus_queue.cc:260] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.208061  7095 ts_tablet_manager.cc:1434] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:27:46.208139  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75c82b1c500f4214aef5d8ed2567f780" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:46.208840  7050 raft_consensus.cc:3060] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.209146  7095 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.199819  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75c82b1c500f4214aef5d8ed2567f780" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
W20260709 12:27:46.211659  6865 leader_election.cc:343] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.200431  7097 consensus_queue.cc:260] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.215890  7097 ts_tablet_manager.cc:1434] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:27:46.215950  7050 raft_consensus.cc:2468] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:46.217245  6866 leader_election.cc:304] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.217530  7097 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:46.203006  7096 raft_consensus.cc:740] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.218941  7095 tablet_bootstrap.cc:654] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.221827  7221 leader_election.cc:290] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.222703  7221 raft_consensus.cc:2804] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:46.223165  7221 raft_consensus.cc:697] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:46.223280  7110 raft_consensus.cc:493] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.223815  7110 raft_consensus.cc:515] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.224561  7097 tablet_bootstrap.cc:654] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.224007  7221 consensus_queue.cc:237] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.221094  7049 raft_consensus.cc:2468] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:46.227052  6866 leader_election.cc:304] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.227727  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82cf9bc8bd54dc08a7e03270cacde99" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:46.228264  7138 raft_consensus.cc:2804] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:46.228788  7138 raft_consensus.cc:697] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
W20260709 12:27:46.229872  6939 leader_election.cc:343] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.229604  7138 consensus_queue.cc:237] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.231702  7110 leader_election.cc:290] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:46.232501  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82cf9bc8bd54dc08a7e03270cacde99" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:46.233372  7096 consensus_queue.cc:260] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
W20260709 12:27:46.236061  6942 leader_election.cc:343] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.236589  6942 leader_election.cc:304] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:46.237655  7110 raft_consensus.cc:2749] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.221158  7094 tablet_bootstrap.cc:654] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.239668  7096 ts_tablet_manager.cc:1434] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
I20260709 12:27:46.248099  7096 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:46.249362  7095 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.250007  7095 ts_tablet_manager.cc:1403] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.017s
I20260709 12:27:46.251840  7097 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:46.252377  7097 ts_tablet_manager.cc:1403] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.007s
I20260709 12:27:46.252952  7095 raft_consensus.cc:359] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.253674  7095 raft_consensus.cc:385] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.253966  7095 raft_consensus.cc:740] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.254756  7095 consensus_queue.cc:260] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.255687  7111 raft_consensus.cc:493] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.255609  7097 raft_consensus.cc:359] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.256525  7097 raft_consensus.cc:385] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.256280  7111 raft_consensus.cc:515] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.256880  7095 ts_tablet_manager.cc:1434] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:27:46.253366  7096 tablet_bootstrap.cc:654] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.257995  7095 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.258467  7111 leader_election.cc:290] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:46.260051  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:46.260421  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:46.261863  6790 leader_election.cc:343] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.262192  7094 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:46.262754  7094 ts_tablet_manager.cc:1403] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.000s
I20260709 12:27:46.264695  7097 raft_consensus.cc:740] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.265807  7095 tablet_bootstrap.cc:654] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.265609  7097 consensus_queue.cc:260] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.265826  7094 raft_consensus.cc:359] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.267159  7094 raft_consensus.cc:385] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:27:46.267273  6793 leader_election.cc:343] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.267647  7094 raft_consensus.cc:740] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.271072  7097 ts_tablet_manager.cc:1434] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 12:27:46.271579  7094 consensus_queue.cc:260] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.267761  6793 leader_election.cc:304] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.276499  7205 raft_consensus.cc:2749] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.269027  6723 catalog_manager.cc:5697] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:46.283628  7094 ts_tablet_manager.cc:1434] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:27:46.290948  7097 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:46.291550  7223 raft_consensus.cc:493] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.292294  7223 raft_consensus.cc:515] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.296109  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:46.297885  7013 leader_election.cc:343] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.300748  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:46.302564  7016 leader_election.cc:343] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.303182  7016 leader_election.cc:304] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:46.304399  7140 raft_consensus.cc:2749] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.303378  6725 catalog_manager.cc:5697] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:46.303212  7094 tablet_bootstrap.cc:492] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:46.316730  7097 tablet_bootstrap.cc:654] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.312091  7223 leader_election.cc:290] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:46.322144  7096 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:46.322830  7096 ts_tablet_manager.cc:1403] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.075s	user 0.022s	sys 0.018s
I20260709 12:27:46.327230  7095 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.327975  7095 ts_tablet_manager.cc:1403] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.070s	user 0.017s	sys 0.041s
I20260709 12:27:46.327260  7096 raft_consensus.cc:359] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.328630  7096 raft_consensus.cc:385] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.328992  7096 raft_consensus.cc:740] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.329339  7097 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:46.330859  7094 tablet_bootstrap.cc:654] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.332963  7097 ts_tablet_manager.cc:1403] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.005s
I20260709 12:27:46.331390  7096 consensus_queue.cc:260] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.330785  7095 raft_consensus.cc:359] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.336079  7097 raft_consensus.cc:359] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.336527  7095 raft_consensus.cc:385] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.336843  7097 raft_consensus.cc:385] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.336987  7095 raft_consensus.cc:740] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.337273  7097 raft_consensus.cc:740] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.337896  7095 consensus_queue.cc:260] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.340193  7095 ts_tablet_manager.cc:1434] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:27:46.342085  7095 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.338074  7097 consensus_queue.cc:260] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.346633  7221 raft_consensus.cc:493] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.347224  7221 raft_consensus.cc:515] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.350627  7221 leader_election.cc:290] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:46.351085  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c77ef2e07ff418b996e1c72415228d6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:46.352715  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c77ef2e07ff418b996e1c72415228d6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.353564  7049 raft_consensus.cc:2468] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:46.355363  6866 leader_election.cc:304] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:46.357636  7110 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.358404  7097 ts_tablet_manager.cc:1434] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:27:46.358127  7110 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.359521  7097 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:46.360714  7110 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:46.361999  7221 raft_consensus.cc:2804] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.362583  7221 raft_consensus.cc:493] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.363029  7221 raft_consensus.cc:3060] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:46.363785  6865 leader_election.cc:343] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.364981  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.345433  7096 ts_tablet_manager.cc:1434] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:27:46.366451  7096 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:46.372849  7096 tablet_bootstrap.cc:654] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:46.366532  6940 leader_election.cc:343] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.375744  7097 tablet_bootstrap.cc:654] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.376549  7095 tablet_bootstrap.cc:654] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.378844  7156 consensus_queue.cc:1048] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:46.381968  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:46.383646  6939 leader_election.cc:343] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.384104  6939 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:46.385234  7156 raft_consensus.cc:2749] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.386775  7111 raft_consensus.cc:493] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.387274  7111 raft_consensus.cc:515] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.389473  7111 leader_election.cc:290] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.390424  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:46.391223  6974 raft_consensus.cc:2468] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:46.392776  6790 leader_election.cc:304] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:46.393886  7111 raft_consensus.cc:2804] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.394305  7111 raft_consensus.cc:493] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.394693  7111 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.395964  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.396589  7049 raft_consensus.cc:2468] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:46.403505  7156 consensus_queue.cc:1048] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.404772  7108 raft_consensus.cc:493] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.405294  7108 raft_consensus.cc:515] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.407657  7108 leader_election.cc:290] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:46.409827  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:46.410362  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:46.411116  6902 raft_consensus.cc:2468] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
W20260709 12:27:46.411381  6939 leader_election.cc:343] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.412747  7094 tablet_bootstrap.cc:492] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:46.413254  7094 ts_tablet_manager.cc:1403] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.125s	user 0.023s	sys 0.014s
I20260709 12:27:46.415108  7097 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:46.415612  7097 ts_tablet_manager.cc:1403] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.056s	user 0.011s	sys 0.004s
I20260709 12:27:46.416109  7094 raft_consensus.cc:359] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.416844  7094 raft_consensus.cc:385] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.416719  6942 leader_election.cc:304] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:46.417335  7094 raft_consensus.cc:740] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.418458  7097 raft_consensus.cc:359] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.419147  7097 raft_consensus.cc:385] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.419322  7095 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.419409  7097 raft_consensus.cc:740] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.419956  7095 ts_tablet_manager.cc:1403] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.078s	user 0.012s	sys 0.002s
I20260709 12:27:46.422399  7108 raft_consensus.cc:2804] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.422839  7108 raft_consensus.cc:493] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.377451  7221 raft_consensus.cc:515] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.423425  7094 consensus_queue.cc:260] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.424764  7097 consensus_queue.cc:260] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.423950  7108 raft_consensus.cc:3060] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.427450  7111 raft_consensus.cc:515] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.430845  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:46.431180  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:46.431419  6974 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.431794  7050 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.422652  7095 raft_consensus.cc:359] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.436527  7097 ts_tablet_manager.cc:1434] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:27:46.436753  7095 raft_consensus.cc:385] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.434758  7111 leader_election.cc:290] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.437137  7095 raft_consensus.cc:740] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.437824  7097 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:46.438369  7095 consensus_queue.cc:260] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.443434  7094 ts_tablet_manager.cc:1434] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.002s
I20260709 12:27:46.447182  7221 leader_election.cc:290] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:46.448201  7095 ts_tablet_manager.cc:1434] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:27:46.448110  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c77ef2e07ff418b996e1c72415228d6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:46.448755  7049 raft_consensus.cc:3060] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.449249  7095 tablet_bootstrap.cc:492] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.452415  6974 raft_consensus.cc:2468] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:46.453661  6790 leader_election.cc:304] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:46.457675  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c77ef2e07ff418b996e1c72415228d6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:46.458320  6827 raft_consensus.cc:3060] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.460623  7097 tablet_bootstrap.cc:654] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.462904  7094 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
W20260709 12:27:46.464690  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:46.470343  7111 raft_consensus.cc:2804] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:46.469515  7221 raft_consensus.cc:493] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.470918  7111 raft_consensus.cc:697] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:46.471033  7221 raft_consensus.cc:515] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.471796  7111 consensus_queue.cc:237] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.475689  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc898afa9a94274850432d379c5203e" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.454614  7050 raft_consensus.cc:2468] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:46.478782  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc898afa9a94274850432d379c5203e" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:46.479908  7221 leader_election.cc:290] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
W20260709 12:27:46.480252  6866 leader_election.cc:343] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:46.481997  6865 leader_election.cc:343] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.482367  7223 consensus_queue.cc:1048] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.483516  7049 raft_consensus.cc:2468] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:46.482538  6865 leader_election.cc:304] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:46.484766  6866 leader_election.cc:304] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:46.485181  7221 raft_consensus.cc:2749] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.485584  7138 raft_consensus.cc:2804] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:46.485703  7095 tablet_bootstrap.cc:654] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.486166  7138 raft_consensus.cc:697] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:46.487118  7138 consensus_queue.cc:237] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.490530  7094 tablet_bootstrap.cc:654] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.494167  7108 raft_consensus.cc:515] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.497911  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:46.507957  7096 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:46.508662  7096 ts_tablet_manager.cc:1403] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.142s	user 0.016s	sys 0.011s
I20260709 12:27:46.502051  6827 raft_consensus.cc:2468] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:46.512107  7108 leader_election.cc:290] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:46.513468  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
W20260709 12:27:46.514700  6939 leader_election.cc:343] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.514086  6902 raft_consensus.cc:3060] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.516999  6725 catalog_manager.cc:5697] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:46.516795  6723 catalog_manager.cc:5697] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:46.522333  7111 consensus_queue.cc:1048] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.533568  6902 raft_consensus.cc:2468] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:46.534996  6942 leader_election.cc:304] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:46.536747  7154 raft_consensus.cc:2804] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:46.537225  7154 raft_consensus.cc:697] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:46.538012  7154 consensus_queue.cc:237] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.541023  7095 tablet_bootstrap.cc:492] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.541498  7095 ts_tablet_manager.cc:1403] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.092s	user 0.024s	sys 0.002s
I20260709 12:27:46.543999  6827 raft_consensus.cc:3060] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.542699  7096 raft_consensus.cc:359] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.544608  7096 raft_consensus.cc:385] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.544893  7096 raft_consensus.cc:740] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.531323  7111 raft_consensus.cc:493] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.545511  7096 consensus_queue.cc:260] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.545783  7111 raft_consensus.cc:515] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.547798  7096 ts_tablet_manager.cc:1434] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260709 12:27:46.548834  7096 tablet_bootstrap.cc:492] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:46.554893  7097 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:46.555439  7096 tablet_bootstrap.cc:654] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.555572  7097 ts_tablet_manager.cc:1403] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.118s	user 0.019s	sys 0.017s
I20260709 12:27:46.555819  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.556506  7049 raft_consensus.cc:2468] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:46.560787  7095 raft_consensus.cc:359] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.565410  7095 raft_consensus.cc:385] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.565692  7095 raft_consensus.cc:740] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.566375  7095 consensus_queue.cc:260] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.569531  7095 ts_tablet_manager.cc:1434] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.001s
I20260709 12:27:46.571108  7138 raft_consensus.cc:493] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.572944  7097 raft_consensus.cc:359] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.561268  6791 leader_election.cc:304] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:46.574052  7095 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.573885  7138 raft_consensus.cc:515] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.576603  6723 catalog_manager.cc:5697] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:46.574847  7097 raft_consensus.cc:385] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.574534  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:46.579803  6902 raft_consensus.cc:2468] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:46.579487  7138 leader_election.cc:290] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:46.580762  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:46.582399  7205 raft_consensus.cc:2804] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.582780  7205 raft_consensus.cc:493] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.583096  7205 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.571424  7111 leader_election.cc:290] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:46.587069  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.588486  6866 leader_election.cc:304] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:46.589161  7095 tablet_bootstrap.cc:654] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.588047  7097 raft_consensus.cc:740] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.597266  7096 tablet_bootstrap.cc:492] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:46.597724  7096 ts_tablet_manager.cc:1403] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.008s
I20260709 12:27:46.598934  7097 consensus_queue.cc:260] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.603941  7096 raft_consensus.cc:359] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.604619  7096 raft_consensus.cc:385] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.604892  7096 raft_consensus.cc:740] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.605592  7096 consensus_queue.cc:260] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.607434  7205 raft_consensus.cc:515] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.608737  7096 ts_tablet_manager.cc:1434] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:27:46.609537  7205 leader_election.cc:290] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:46.610132  7096 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:46.598548  7138 raft_consensus.cc:2749] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.616906  6902 raft_consensus.cc:3060] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.617347  7096 tablet_bootstrap.cc:654] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.617923  7223 consensus_queue.cc:1048] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.622953  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:46.623587  6901 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.624791  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:46.625366  7050 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.637863  7097 ts_tablet_manager.cc:1434] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.082s	user 0.011s	sys 0.001s
I20260709 12:27:46.638976  7097 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:46.641578  7154 raft_consensus.cc:493] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.642263  7154 raft_consensus.cc:515] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.643136  7094 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:46.643661  7094 ts_tablet_manager.cc:1403] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.181s	user 0.008s	sys 0.010s
I20260709 12:27:46.645630  7097 tablet_bootstrap.cc:654] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.647215  7154 leader_election.cc:290] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.648499  7094 raft_consensus.cc:359] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.650233  6974 raft_consensus.cc:3060] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.651011  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.655223  7094 raft_consensus.cc:385] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.658051  7049 raft_consensus.cc:2393] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 1: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:46.659175  7095 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.659643  7095 ts_tablet_manager.cc:1403] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.086s	user 0.011s	sys 0.008s
I20260709 12:27:46.656471  7094 raft_consensus.cc:740] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.662354  7095 raft_consensus.cc:359] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.663090  7095 raft_consensus.cc:385] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.663403  7095 raft_consensus.cc:740] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.664947  7095 consensus_queue.cc:260] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.668306  7205 consensus_queue.cc:1048] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.670258  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:46.665994  6901 raft_consensus.cc:2468] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:46.662091  7050 raft_consensus.cc:2468] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:46.673463  6791 leader_election.cc:304] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:46.674544  7205 raft_consensus.cc:2804] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:46.675014  7205 raft_consensus.cc:697] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:46.675818  7205 consensus_queue.cc:237] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.676287  7221 consensus_queue.cc:1048] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.678038  6942 leader_election.cc:304] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:46.680219  7138 raft_consensus.cc:493] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.680732  7138 raft_consensus.cc:515] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.682782  7138 leader_election.cc:290] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:46.684890  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82cf9bc8bd54dc08a7e03270cacde99" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:46.685583  6976 raft_consensus.cc:2468] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:46.686969  6865 leader_election.cc:304] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:46.688369  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82cf9bc8bd54dc08a7e03270cacde99" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:46.694115  6865 leader_election.cc:343] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.694427  7094 consensus_queue.cc:260] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.695881  7110 raft_consensus.cc:493] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.696498  7138 raft_consensus.cc:2804] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.696400  7110 raft_consensus.cc:515] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.696893  7138 raft_consensus.cc:493] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.697290  7138 raft_consensus.cc:3060] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.698521  7110 leader_election.cc:290] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.699740  7094 ts_tablet_manager.cc:1434] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.003s
I20260709 12:27:46.700826  7094 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:46.703369  7095 ts_tablet_manager.cc:1434] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 12:27:46.703732  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75c82b1c500f4214aef5d8ed2567f780" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.704437  7095 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:46.707990  7154 raft_consensus.cc:2749] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.708484  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75c82b1c500f4214aef5d8ed2567f780" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:46.712630  7221 consensus_queue.cc:1048] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.714157  6942 leader_election.cc:304] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:46.715425  7094 tablet_bootstrap.cc:654] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.720801  7108 raft_consensus.cc:2749] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.734889  6976 raft_consensus.cc:3060] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.744896  7096 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:46.745445  7096 ts_tablet_manager.cc:1403] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.136s	user 0.041s	sys 0.031s
I20260709 12:27:46.751077  7221 consensus_queue.cc:1048] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.756893  7181 raft_consensus.cc:493] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.757612  7181 raft_consensus.cc:515] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.759573  7097 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:46.760071  7097 ts_tablet_manager.cc:1403] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.121s	user 0.007s	sys 0.012s
I20260709 12:27:46.761085  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93204a5a0d0742ad9cd21a8a280af392" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:46.762104  7181 leader_election.cc:290] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:46.762324  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93204a5a0d0742ad9cd21a8a280af392" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:46.762578  7013 leader_election.cc:343] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.763041  6827 raft_consensus.cc:2468] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:46.764420  7013 leader_election.cc:304] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.765244  7181 raft_consensus.cc:2804] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.765710  7181 raft_consensus.cc:493] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.766090  7181 raft_consensus.cc:3060] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.758133  7138 raft_consensus.cc:515] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.772539  7221 consensus_queue.cc:1048] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.775616  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82cf9bc8bd54dc08a7e03270cacde99" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
W20260709 12:27:46.777164  6865 leader_election.cc:343] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.785141  7097 raft_consensus.cc:359] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.787669  7181 raft_consensus.cc:515] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.787305  6724 catalog_manager.cc:5697] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:46.790444  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93204a5a0d0742ad9cd21a8a280af392" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:46.790699  7138 leader_election.cc:290] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:46.790591  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82cf9bc8bd54dc08a7e03270cacde99" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:46.790616  7095 tablet_bootstrap.cc:654] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.791430  6976 raft_consensus.cc:3060] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:46.791693  7013 leader_election.cc:343] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.794757  7097 raft_consensus.cc:385] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.795087  7097 raft_consensus.cc:740] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.785284  7096 raft_consensus.cc:359] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.796389  7096 raft_consensus.cc:385] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.796679  7096 raft_consensus.cc:740] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.797631  7096 consensus_queue.cc:260] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.800204  7096 ts_tablet_manager.cc:1434] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 12:27:46.801281  7096 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:46.805302  7181 leader_election.cc:290] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:46.805833  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93204a5a0d0742ad9cd21a8a280af392" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:46.806478  6824 raft_consensus.cc:3060] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.811923  7097 consensus_queue.cc:260] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.813977  7097 ts_tablet_manager.cc:1434] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.001s
I20260709 12:27:46.805871  7181 raft_consensus.cc:493] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.815032  7181 raft_consensus.cc:515] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.815311  7140 raft_consensus.cc:493] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.815793  7140 raft_consensus.cc:515] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.817764  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c78f0ec1af4414a7e52c69f19b6c27" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:46.818296  7181 leader_election.cc:290] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
W20260709 12:27:46.819571  7013 leader_election.cc:343] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.817899  7140 leader_election.cc:290] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:46.819855  7097 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:46.819784  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbf51a7748b461686ff7db762bba509" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:46.821233  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c78f0ec1af4414a7e52c69f19b6c27" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:46.817744  6824 raft_consensus.cc:2468] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:46.823169  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbf51a7748b461686ff7db762bba509" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:46.824681  7016 leader_election.cc:343] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.825233  7094 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:46.825765  7094 ts_tablet_manager.cc:1403] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.125s	user 0.010s	sys 0.007s
W20260709 12:27:46.826447  7013 leader_election.cc:343] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.826905  7097 tablet_bootstrap.cc:654] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.826907  7013 leader_election.cc:304] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.829137  7140 raft_consensus.cc:2749] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.830435  7094 raft_consensus.cc:359] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.841671  7094 raft_consensus.cc:385] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.831554  7274 raft_consensus.cc:493] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.842061  7094 raft_consensus.cc:740] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.842597  7274 raft_consensus.cc:515] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:46.842998  7094 consensus_queue.cc:260] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.845609  7274 leader_election.cc:290] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:46.845773  7094 ts_tablet_manager.cc:1434] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.001s
I20260709 12:27:46.847486  7094 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:46.847812  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:46.848747  7205 raft_consensus.cc:493] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.849218  7205 raft_consensus.cc:515] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.850510  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.852563  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:46.853386  7205 leader_election.cc:290] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:46.853655  6901 raft_consensus.cc:2393] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:46.829082  7013 leader_election.cc:304] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.834609  6976 raft_consensus.cc:2468] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:46.848114  7108 raft_consensus.cc:493] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.858381  7108 raft_consensus.cc:515] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.858685  7140 raft_consensus.cc:2804] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:27:46.858765  7013 leader_election.cc:343] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.859321  7013 leader_election.cc:304] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.833784  7096 tablet_bootstrap.cc:654] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.860574  7108 leader_election.cc:290] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:46.860666  7181 raft_consensus.cc:2749] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:46.859377  6865 leader_election.cc:343] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.859342  7140 raft_consensus.cc:697] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:46.862720  7094 tablet_bootstrap.cc:654] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:46.862769  7140 consensus_queue.cc:237] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:46.866627  7097 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:46.867066  7097 ts_tablet_manager.cc:1403] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.047s	user 0.026s	sys 0.012s
W20260709 12:27:46.868716  6866 leader_election.cc:343] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:46.868955  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93204a5a0d0742ad9cd21a8a280af392" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:46.869781  7097 raft_consensus.cc:359] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.870486  7097 raft_consensus.cc:385] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:46.870318  6865 leader_election.cc:304] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:46.870783  7097 raft_consensus.cc:740] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:46.869179  6866 leader_election.cc:304] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.871662  7274 raft_consensus.cc:2804] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:46.871438  7097 consensus_queue.cc:260] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:46.872769  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:46.872795  7222 raft_consensus.cc:2749] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.876261  6790 leader_election.cc:304] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:46.872133  7274 raft_consensus.cc:697] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:46.877976  7274 consensus_queue.cc:237] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.877086  7111 raft_consensus.cc:3060] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.880604  7097 ts_tablet_manager.cc:1434] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:27:46.889443  7156 raft_consensus.cc:493] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.889946  7156 raft_consensus.cc:515] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.896229  7156 leader_election.cc:290] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:46.898106  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:46.898622  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93204a5a0d0742ad9cd21a8a280af392" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:46.898820  6827 raft_consensus.cc:2468] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:46.899382  6824 raft_consensus.cc:2393] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 1: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:46.900242  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:46.900889  6901 raft_consensus.cc:2468] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:46.901674  7263 consensus_queue.cc:1048] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.910786  7097 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:46.916868  6939 leader_election.cc:304] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:46.917833  6939 leader_election.cc:304] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:46.919183  7108 raft_consensus.cc:2804] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:46.919687  7108 raft_consensus.cc:493] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.920125  7108 raft_consensus.cc:3060] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.921785  7154 raft_consensus.cc:3060] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.924140  6723 catalog_manager.cc:5697] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:46.927964  7138 consensus_queue.cc:1048] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:46.931263  7154 raft_consensus.cc:2749] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.931862  7263 consensus_queue.cc:1048] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:46.912020  7111 raft_consensus.cc:2749] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:46.962513  6724 catalog_manager.cc:5697] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:46.978077  7095 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:46.978837  7095 ts_tablet_manager.cc:1403] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.275s	user 0.027s	sys 0.007s
I20260709 12:27:46.989096  7108 raft_consensus.cc:515] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:46.992601  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:46.993412  6824 raft_consensus.cc:3060] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:46.995918  7097 tablet_bootstrap.cc:654] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.001909  7094 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:47.002437  7094 ts_tablet_manager.cc:1403] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.155s	user 0.025s	sys 0.031s
I20260709 12:27:47.007709  7094 raft_consensus.cc:359] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.008349  7094 raft_consensus.cc:385] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.008594  7094 raft_consensus.cc:740] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.009305  7094 consensus_queue.cc:260] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.016520  6824 raft_consensus.cc:2468] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:47.029080  6939 leader_election.cc:304] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.019481  7096 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:47.031289  7096 ts_tablet_manager.cc:1403] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.230s	user 0.046s	sys 0.026s
I20260709 12:27:47.031544  7110 raft_consensus.cc:2804] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:47.032107  7110 raft_consensus.cc:697] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:47.032933  7110 consensus_queue.cc:237] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.036741  7096 raft_consensus.cc:359] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.037680  7096 raft_consensus.cc:385] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.037215  7138 consensus_queue.cc:1048] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:47.038295  7096 raft_consensus.cc:740] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.039743  7096 consensus_queue.cc:260] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.043687  7095 raft_consensus.cc:359] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.020004  7094 ts_tablet_manager.cc:1434] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.002s
I20260709 12:27:47.061292  7094 tablet_bootstrap.cc:492] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:47.066816  7094 tablet_bootstrap.cc:654] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.045986  7108 leader_election.cc:290] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:47.047325  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:47.068054  6901 raft_consensus.cc:3060] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.070099  7095 raft_consensus.cc:385] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.070447  7095 raft_consensus.cc:740] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.071117  7095 consensus_queue.cc:260] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.073233  7095 ts_tablet_manager.cc:1434] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.087s	user 0.004s	sys 0.000s
I20260709 12:27:47.074209  7095 tablet_bootstrap.cc:492] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:47.075593  6901 raft_consensus.cc:2468] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:47.049824  7096 ts_tablet_manager.cc:1434] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:27:47.060024  7097 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:47.078262  7097 ts_tablet_manager.cc:1403] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.172s	user 0.026s	sys 0.029s
I20260709 12:27:47.079950  7095 tablet_bootstrap.cc:654] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.081036  7097 raft_consensus.cc:359] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.081759  7097 raft_consensus.cc:385] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.082029  7097 raft_consensus.cc:740] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.082895  7097 consensus_queue.cc:260] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.086239  7097 ts_tablet_manager.cc:1434] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:27:47.087491  7097 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:47.091270  7096 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:47.092594  7094 tablet_bootstrap.cc:492] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:47.093120  7094 ts_tablet_manager.cc:1403] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:27:47.068042  6723 catalog_manager.cc:5697] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:47.095728  7094 raft_consensus.cc:359] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.096385  7094 raft_consensus.cc:385] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.096650  7094 raft_consensus.cc:740] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.098274  7094 consensus_queue.cc:260] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.100450  7094 ts_tablet_manager.cc:1434] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:27:47.101466  7094 tablet_bootstrap.cc:492] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:47.101897  7096 tablet_bootstrap.cc:654] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.107254  7094 tablet_bootstrap.cc:654] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.112149  7111 raft_consensus.cc:493] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.112849  7096 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:47.112643  7111 raft_consensus.cc:515] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.113343  7096 ts_tablet_manager.cc:1403] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.001s
I20260709 12:27:47.114804  7097 tablet_bootstrap.cc:654] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.114773  7111 leader_election.cc:290] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:47.116295  7095 tablet_bootstrap.cc:492] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:47.116321  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82cf9bc8bd54dc08a7e03270cacde99" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:47.116948  7095 ts_tablet_manager.cc:1403] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.021s
I20260709 12:27:47.117734  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d82cf9bc8bd54dc08a7e03270cacde99" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.118089  7108 raft_consensus.cc:493] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.118537  6975 raft_consensus.cc:2393] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:47.118654  7108 raft_consensus.cc:515] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.119611  7095 raft_consensus.cc:359] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.120308  7095 raft_consensus.cc:385] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.120577  7095 raft_consensus.cc:740] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.119838  6790 leader_election.cc:304] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:47.121066  7108 leader_election.cc:290] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.121277  7111 raft_consensus.cc:3060] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.121232  7095 consensus_queue.cc:260] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.123481  7095 ts_tablet_manager.cc:1434] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:27:47.124573  7095 tablet_bootstrap.cc:492] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:47.125830  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbf51a7748b461686ff7db762bba509" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:47.126541  6901 raft_consensus.cc:2468] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:47.127880  6942 leader_election.cc:304] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.128656  7154 raft_consensus.cc:2804] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.129026  7154 raft_consensus.cc:493] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.129402  7154 raft_consensus.cc:3060] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.130880  7111 raft_consensus.cc:2749] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.131135  7096 raft_consensus.cc:359] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.130928  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbf51a7748b461686ff7db762bba509" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:47.131824  7096 raft_consensus.cc:385] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.132165  7095 tablet_bootstrap.cc:654] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.132135  7050 raft_consensus.cc:2468] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:47.136904  7154 raft_consensus.cc:515] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.137784  7094 tablet_bootstrap.cc:492] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:47.132112  7096 raft_consensus.cc:740] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.138365  7094 ts_tablet_manager.cc:1403] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.004s
I20260709 12:27:47.139055  7096 consensus_queue.cc:260] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.141513  7096 ts_tablet_manager.cc:1434] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 12:27:47.141780  7097 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:47.142292  7097 ts_tablet_manager.cc:1403] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.006s
I20260709 12:27:47.142565  7096 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:47.141198  7094 raft_consensus.cc:359] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.145478  7094 raft_consensus.cc:385] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.145756  7094 raft_consensus.cc:740] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.146692  7094 consensus_queue.cc:260] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.150605  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbf51a7748b461686ff7db762bba509" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:47.151386  7050 raft_consensus.cc:3060] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.152858  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfbf51a7748b461686ff7db762bba509" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:47.153416  6901 raft_consensus.cc:3060] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.154486  7094 ts_tablet_manager.cc:1434] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:27:47.155543  7094 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:47.158447  7108 consensus_queue.cc:1048] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:47.163826  7110 consensus_queue.cc:1048] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:47.169502  7096 tablet_bootstrap.cc:654] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.175681  7094 tablet_bootstrap.cc:654] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.183748  7154 leader_election.cc:290] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.188160  7094 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:47.188674  7094 ts_tablet_manager.cc:1403] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.000s
I20260709 12:27:47.191442  7094 raft_consensus.cc:359] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.192070  7094 raft_consensus.cc:385] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.192348  7094 raft_consensus.cc:740] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.198565  7094 consensus_queue.cc:260] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.206809  6901 raft_consensus.cc:2468] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:47.208051  6942 leader_election.cc:304] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.208897  7111 consensus_queue.cc:1048] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:47.208251  7094 ts_tablet_manager.cc:1434] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:27:47.211179  7094 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:47.149379  7097 raft_consensus.cc:359] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.212064  7097 raft_consensus.cc:385] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.212347  7097 raft_consensus.cc:740] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.213001  7097 consensus_queue.cc:260] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.208942  7154 raft_consensus.cc:2804] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:47.223037  7094 tablet_bootstrap.cc:654] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.227489  7154 raft_consensus.cc:697] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:47.229422  7154 consensus_queue.cc:237] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.186317  7050 raft_consensus.cc:2468] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:47.241791  7097 ts_tablet_manager.cc:1434] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.000s
I20260709 12:27:47.256179  7097 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:47.270061  7094 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:47.270642  7094 ts_tablet_manager.cc:1403] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.000s
I20260709 12:27:47.279114  7096 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:47.279665  7096 ts_tablet_manager.cc:1403] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.137s	user 0.012s	sys 0.023s
I20260709 12:27:47.280717  7094 raft_consensus.cc:359] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.281425  7094 raft_consensus.cc:385] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.281737  7094 raft_consensus.cc:740] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.282524  7094 consensus_queue.cc:260] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.284627  7095 tablet_bootstrap.cc:492] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:47.285104  7094 ts_tablet_manager.cc:1434] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:27:47.282588  7096 raft_consensus.cc:359] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.285259  7095 ts_tablet_manager.cc:1403] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.161s	user 0.049s	sys 0.091s
I20260709 12:27:47.285604  7096 raft_consensus.cc:385] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.287081  7094 tablet_bootstrap.cc:492] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:47.288137  7095 raft_consensus.cc:359] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.288797  7095 raft_consensus.cc:385] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.289037  7095 raft_consensus.cc:740] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.285991  7096 raft_consensus.cc:740] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.289655  7095 consensus_queue.cc:260] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.291260  7096 consensus_queue.cc:260] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.296068  7095 ts_tablet_manager.cc:1434] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:27:47.296187  7110 raft_consensus.cc:493] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.297183  7095 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:47.297323  7097 tablet_bootstrap.cc:654] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.296774  7110 raft_consensus.cc:515] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.300716  7205 consensus_queue.cc:1048] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:47.302201  7110 leader_election.cc:290] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:47.303335  7222 consensus_queue.cc:1048] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:47.303841  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c78f0ec1af4414a7e52c69f19b6c27" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:47.304389  7095 tablet_bootstrap.cc:654] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:47.305372  6939 leader_election.cc:343] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.296368  7096 ts_tablet_manager.cc:1434] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:27:47.308439  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c78f0ec1af4414a7e52c69f19b6c27" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:47.309345  7049 raft_consensus.cc:2468] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:47.310491  7096 tablet_bootstrap.cc:492] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:47.313460  7274 consensus_queue.cc:1048] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:47.313452  6940 leader_election.cc:304] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:47.315665  7154 raft_consensus.cc:2804] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.316035  7154 raft_consensus.cc:493] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.316394  7154 raft_consensus.cc:3060] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.318686  7263 raft_consensus.cc:493] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.319335  7263 raft_consensus.cc:515] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.326006  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:47.328008  7181 consensus_queue.cc:1048] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:47.322067  7263 leader_election.cc:290] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
W20260709 12:27:47.333894  6793 leader_election.cc:343] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.344316  7094 tablet_bootstrap.cc:654] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.343519  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:47.330363  7154 raft_consensus.cc:515] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.356971  7156 raft_consensus.cc:493] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.366129  7156 raft_consensus.cc:515] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.367579  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c78f0ec1af4414a7e52c69f19b6c27" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:47.368320  7050 raft_consensus.cc:3060] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.368819  7156 leader_election.cc:290] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.369469  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:47.362854  7096 tablet_bootstrap.cc:654] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.370342  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:47.370653  6901 raft_consensus.cc:2468] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
W20260709 12:27:47.371649  6940 leader_election.cc:343] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.372331  6942 leader_election.cc:304] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:47.375231  7181 raft_consensus.cc:493] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.375748  7181 raft_consensus.cc:515] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.378515  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "122d63cf583043458b719df07a6a694d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.379387  6974 raft_consensus.cc:2468] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:47.380225  6825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "122d63cf583043458b719df07a6a694d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:47.381223  7013 leader_election.cc:304] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
W20260709 12:27:47.382539  7013 leader_election.cc:343] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.383354  7223 raft_consensus.cc:2804] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.383884  7223 raft_consensus.cc:493] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.384301  7223 raft_consensus.cc:3060] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.341938  6724 catalog_manager.cc:5697] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:47.391482  6791 leader_election.cc:343] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.392042  7096 tablet_bootstrap.cc:492] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:47.392982  6791 leader_election.cc:304] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:47.394181  7263 raft_consensus.cc:2749] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.393479  7096 ts_tablet_manager.cc:1403] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.085s	user 0.013s	sys 0.002s
I20260709 12:27:47.381520  7181 leader_election.cc:290] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:47.397425  7096 raft_consensus.cc:359] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.378482  7156 raft_consensus.cc:2804] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.404997  7156 raft_consensus.cc:493] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.398160  7096 raft_consensus.cc:385] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.407176  7096 raft_consensus.cc:740] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.407397  7156 raft_consensus.cc:3060] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.407977  7096 consensus_queue.cc:260] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.413530  7096 ts_tablet_manager.cc:1434] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 12:27:47.414608  7096 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:47.423991  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c78f0ec1af4414a7e52c69f19b6c27" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:47.424548  7110 consensus_queue.cc:1048] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:27:47.425591  6939 leader_election.cc:343] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.427815  7154 leader_election.cc:290] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:47.429841  7050 raft_consensus.cc:2468] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:47.431100  6940 leader_election.cc:304] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:47.431800  7094 tablet_bootstrap.cc:492] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:47.432309  7094 ts_tablet_manager.cc:1403] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.145s	user 0.013s	sys 0.007s
I20260709 12:27:47.433264  7110 raft_consensus.cc:2804] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:47.433807  7110 raft_consensus.cc:697] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:47.434995  7110 consensus_queue.cc:237] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.442108  7223 raft_consensus.cc:515] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.445289  7223 leader_election.cc:290] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:47.459379  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "122d63cf583043458b719df07a6a694d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:47.459995  6974 raft_consensus.cc:3060] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.463951  7223 raft_consensus.cc:493] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.464552  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "122d63cf583043458b719df07a6a694d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:47.464710  7223 raft_consensus.cc:515] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.469798  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc898afa9a94274850432d379c5203e" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:47.470614  6901 raft_consensus.cc:2468] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:47.471681  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc898afa9a94274850432d379c5203e" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:47.472326  6827 raft_consensus.cc:2468] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:47.472235  7016 leader_election.cc:304] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:47.473382  7223 leader_election.cc:290] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
W20260709 12:27:47.474924  7013 leader_election.cc:343] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.475410  7223 raft_consensus.cc:2804] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.475790  7223 raft_consensus.cc:493] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.476125  7223 raft_consensus.cc:3060] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.477581  7222 raft_consensus.cc:493] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.480134  7222 raft_consensus.cc:515] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.480350  7181 consensus_queue.cc:1048] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:47.482192  7222 leader_election.cc:290] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.482947  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.483747  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
W20260709 12:27:47.484961  6865 leader_election.cc:343] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:47.485958  6866 leader_election.cc:343] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.486424  6866 leader_election.cc:304] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:47.485975  7263 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.487860  7263 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.488502  7096 tablet_bootstrap.cc:654] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.489959  7263 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.504012  7181 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.504554  7181 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.506556  7181 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:47.508333  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.509189  6976 raft_consensus.cc:2468] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:47.511250  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:47.511823  6827 raft_consensus.cc:2468] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:47.513018  7013 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:47.514390  7181 raft_consensus.cc:2804] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.514817  7181 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.515406  7181 raft_consensus.cc:3060] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.516603  7222 raft_consensus.cc:2749] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.517800  6724 catalog_manager.cc:5697] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:47.534004  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:47.524700  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.535041  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:47.537953  7094 raft_consensus.cc:359] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.539160  7094 raft_consensus.cc:385] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.539671  7094 raft_consensus.cc:740] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.548534  6976 raft_consensus.cc:2468] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:47.554420  6790 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.555300  7263 raft_consensus.cc:2804] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.555663  7263 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.555979  7263 raft_consensus.cc:3060] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.561425  7222 raft_consensus.cc:493] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.561911  7222 raft_consensus.cc:515] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.563972  7222 leader_election.cc:290] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:47.565310  7263 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.568171  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:47.568696  6976 raft_consensus.cc:3060] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.568848  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:47.575621  6976 raft_consensus.cc:2468] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
W20260709 12:27:47.575775  6865 leader_election.cc:343] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.576182  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.577292  7263 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.577744  6790 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.579464  6974 raft_consensus.cc:2468] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
W20260709 12:27:47.579641  6865 leader_election.cc:343] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.580587  6865 leader_election.cc:304] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:47.580662  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:47.580914  7013 leader_election.cc:304] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:47.581799  7222 raft_consensus.cc:2749] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.579545  7205 raft_consensus.cc:2804] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:47.583031  7205 raft_consensus.cc:697] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:47.583886  7205 consensus_queue.cc:237] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.584695  7318 consensus_queue.cc:1048] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:47.560062  7094 consensus_queue.cc:260] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.593340  7094 ts_tablet_manager.cc:1434] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.161s	user 0.007s	sys 0.003s
I20260709 12:27:47.594372  7094 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:47.594796  7096 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:47.595248  7096 ts_tablet_manager.cc:1403] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.181s	user 0.016s	sys 0.014s
I20260709 12:27:47.601114  7223 raft_consensus.cc:515] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.603915  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc898afa9a94274850432d379c5203e" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:47.604182  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc898afa9a94274850432d379c5203e" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:47.604493  6827 raft_consensus.cc:3060] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.604728  6901 raft_consensus.cc:3060] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.605749  7097 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:47.589542  7140 raft_consensus.cc:2804] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:47.606236  7097 ts_tablet_manager.cc:1403] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.350s	user 0.029s	sys 0.026s
I20260709 12:27:47.606297  7140 raft_consensus.cc:697] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:47.607267  7140 consensus_queue.cc:237] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.609005  7097 raft_consensus.cc:359] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.609664  7097 raft_consensus.cc:385] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.609921  7097 raft_consensus.cc:740] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.611382  7097 consensus_queue.cc:260] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.618211  6827 raft_consensus.cc:2468] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:47.621712  7013 leader_election.cc:304] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:47.622184  7322 raft_consensus.cc:493] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.622560  7097 ts_tablet_manager.cc:1434] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:27:47.617558  7223 leader_election.cc:290] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:47.627550  7156 raft_consensus.cc:515] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.631049  7097 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:47.635010  7095 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:47.635655  7095 ts_tablet_manager.cc:1403] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.339s	user 0.056s	sys 0.126s
I20260709 12:27:47.633160  7318 raft_consensus.cc:493] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.637523  7318 raft_consensus.cc:515] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.639143  7095 raft_consensus.cc:359] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.639799  7095 raft_consensus.cc:385] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.640051  7095 raft_consensus.cc:740] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.640553  7312 raft_consensus.cc:2804] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:47.640704  7095 consensus_queue.cc:260] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.624744  7322 raft_consensus.cc:515] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.644306  7322 leader_election.cc:290] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:47.645074  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:47.645717  6900 raft_consensus.cc:2468] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:47.646983  7016 leader_election.cc:304] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:47.641300  7312 raft_consensus.cc:697] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:47.645779  7318 leader_election.cc:290] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:47.635628  6901 raft_consensus.cc:2468] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:47.656425  7096 raft_consensus.cc:359] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.657375  7096 raft_consensus.cc:385] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.657719  7096 raft_consensus.cc:740] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.663708  7181 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.665621  7181 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:47.666795  7050 raft_consensus.cc:2393] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:47.669140  7096 consensus_queue.cc:260] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.676299  7095 ts_tablet_manager.cc:1434] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 12:27:47.682426  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:47.683043  6901 raft_consensus.cc:3060] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.684571  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.685371  6975 raft_consensus.cc:2393] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 1: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:47.687265  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:47.688011  6975 raft_consensus.cc:2393] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 1: Already voted for candidate 38b3b964f82644c7b654e8042953a74e in this term.
I20260709 12:27:47.690616  7156 leader_election.cc:290] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.693440  7097 tablet_bootstrap.cc:654] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.661916  7322 raft_consensus.cc:2804] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.695438  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:47.693997  6901 raft_consensus.cc:2468] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:47.691771  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:47.701678  7096 ts_tablet_manager.cc:1434] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.106s	user 0.007s	sys 0.002s
I20260709 12:27:47.705696  6942 leader_election.cc:304] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.701905  6724 catalog_manager.cc:5697] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:47.707144  7096 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:47.678507  7312 consensus_queue.cc:237] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:27:47.710108  6939 leader_election.cc:343] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.715135  7094 tablet_bootstrap.cc:654] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.710467  7318 raft_consensus.cc:2804] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:47.715883  7318 raft_consensus.cc:697] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:47.716773  7318 consensus_queue.cc:237] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.706465  7322 raft_consensus.cc:493] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.718477  7322 raft_consensus.cc:3060] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.714334  7013 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:47.709359  7095 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:47.727775  7140 raft_consensus.cc:2749] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.742065  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:47.747802  7096 tablet_bootstrap.cc:654] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.754953  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:47.765364  7110 consensus_queue.cc:1048] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:47.767549  7094 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:47.768046  7094 ts_tablet_manager.cc:1403] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.174s	user 0.006s	sys 0.016s
W20260709 12:27:47.771579  6940 leader_election.cc:343] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.772074  6940 leader_election.cc:304] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:47.773252  7154 raft_consensus.cc:2749] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.775070  7322 raft_consensus.cc:515] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.776245  7050 raft_consensus.cc:2393] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 1: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:47.776774  7154 consensus_queue.cc:1048] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:47.759521  7156 raft_consensus.cc:493] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.779698  6724 catalog_manager.cc:5697] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:47.779458  7156 raft_consensus.cc:515] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.794684  7322 leader_election.cc:290] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:47.795887  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:47.796645  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2acfad3040b40b69d4e0262179cdf6e" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:47.796945  6900 raft_consensus.cc:2393] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 1: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:47.804774  7095 tablet_bootstrap.cc:654] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.805063  7013 leader_election.cc:304] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:47.808223  7322 raft_consensus.cc:2749] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.813138  7274 raft_consensus.cc:493] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.813635  7274 raft_consensus.cc:515] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.815464  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:47.816134  6899 raft_consensus.cc:2468] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:47.816574  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:47.817332  6942 leader_election.cc:304] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
W20260709 12:27:47.817968  6865 leader_election.cc:343] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.818231  7154 raft_consensus.cc:2804] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.818635  7154 raft_consensus.cc:493] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.818987  7154 raft_consensus.cc:3060] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.819065  7274 leader_election.cc:290] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:47.820624  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:47.821971  6865 leader_election.cc:343] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.822386  6865 leader_election.cc:304] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:47.823375  7274 raft_consensus.cc:2749] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.824095  7156 leader_election.cc:290] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.824257  6724 catalog_manager.cc:5697] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:47.826066  6939 consensus_peers.cc:597] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:47.827935  7094 raft_consensus.cc:359] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.838145  7094 raft_consensus.cc:385] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.840348  7094 raft_consensus.cc:740] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.841303  7094 consensus_queue.cc:260] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.850055  7094 ts_tablet_manager.cc:1434] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.001s
I20260709 12:27:47.826707  6724 catalog_manager.cc:5697] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:47.832557  7154 raft_consensus.cc:515] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.863134  7154 leader_election.cc:290] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:47.866133  7096 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:47.866641  7096 ts_tablet_manager.cc:1403] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.161s	user 0.040s	sys 0.046s
I20260709 12:27:47.869692  7096 raft_consensus.cc:359] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.870368  7096 raft_consensus.cc:385] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.870616  7096 raft_consensus.cc:740] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.871227  7096 consensus_queue.cc:260] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.873375  7096 ts_tablet_manager.cc:1434] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:27:47.874358  7096 tablet_bootstrap.cc:492] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
W20260709 12:27:47.845243  7330 log.cc:927] Time spent T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:47.836853  7097 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:47.850939  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:47.878044  7095 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:47.878516  7263 raft_consensus.cc:493] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.878710  7097 ts_tablet_manager.cc:1403] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.250s	user 0.017s	sys 0.009s
I20260709 12:27:47.879079  7263 raft_consensus.cc:515] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.880010  7096 tablet_bootstrap.cc:654] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.880381  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:47.880968  6899 raft_consensus.cc:3060] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.878551  7095 ts_tablet_manager.cc:1403] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.171s	user 0.011s	sys 0.005s
I20260709 12:27:47.882647  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.883288  6975 raft_consensus.cc:2468] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:47.884505  6790 leader_election.cc:304] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.885344  7111 raft_consensus.cc:2804] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:47.885697  7111 raft_consensus.cc:493] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.886008  7111 raft_consensus.cc:3060] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.890514  6899 raft_consensus.cc:2468] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:47.891503  7095 raft_consensus.cc:359] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.892208  7095 raft_consensus.cc:385] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.892474  7095 raft_consensus.cc:740] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.893006  7096 tablet_bootstrap.cc:492] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:47.893499  7096 ts_tablet_manager.cc:1403] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.009s
I20260709 12:27:47.893152  7095 consensus_queue.cc:260] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:47.894358  7322 raft_consensus.cc:493] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.894848  7322 raft_consensus.cc:515] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.895192  7111 raft_consensus.cc:515] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.896852  7111 leader_election.cc:290] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:47.897006  7322 leader_election.cc:290] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:47.897637  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:47.898041  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:47.898314  6899 raft_consensus.cc:2468] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:47.898404  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:47.898622  6942 leader_election.cc:304] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.897930  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
W20260709 12:27:47.899497  7013 leader_election.cc:343] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.899669  6975 raft_consensus.cc:3060] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.899842  7154 raft_consensus.cc:2804] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:27:47.900220  7013 leader_election.cc:343] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.900437  7154 raft_consensus.cc:697] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:47.900672  7013 leader_election.cc:304] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:47.901417  7322 raft_consensus.cc:2749] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:47.901320  7154 consensus_queue.cc:237] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.896389  7096 raft_consensus.cc:359] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.902392  7096 raft_consensus.cc:385] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.902673  7096 raft_consensus.cc:740] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.903432  7096 consensus_queue.cc:260] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:47.903739  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:47.904425  6899 raft_consensus.cc:3060] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.891564  7263 leader_election.cc:290] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:47.905668  7096 ts_tablet_manager.cc:1434] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:27:47.906695  7096 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:47.906564  6975 raft_consensus.cc:2468] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:47.907374  7097 raft_consensus.cc:359] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.907809  6790 leader_election.cc:304] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:47.908075  7097 raft_consensus.cc:385] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.908170  7094 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:47.906579  7095 ts_tablet_manager.cc:1434] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 12:27:47.909482  7095 tablet_bootstrap.cc:492] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:47.912892  6899 raft_consensus.cc:2468] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:47.913084  7096 tablet_bootstrap.cc:654] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.925464  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:47.926129  7048 raft_consensus.cc:3060] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:47.934190  7094 tablet_bootstrap.cc:654] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.914395  7263 raft_consensus.cc:2804] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:27:47.917125  6940 leader_election.cc:343] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:47.939756  7263 raft_consensus.cc:697] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:47.940991  7263 consensus_queue.cc:237] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.946026  7094 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:47.946641  7094 ts_tablet_manager.cc:1403] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.003s
I20260709 12:27:47.908396  7097 raft_consensus.cc:740] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.952273  7097 consensus_queue.cc:260] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:47.957371  6723 catalog_manager.cc:5697] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:47.951120  7048 raft_consensus.cc:2468] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:47.953238  7154 consensus_queue.cc:1048] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:47.960211  7097 ts_tablet_manager.cc:1434] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.081s	user 0.005s	sys 0.001s
I20260709 12:27:47.957690  7095 tablet_bootstrap.cc:654] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.969408  6724 catalog_manager.cc:5697] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:47.949198  7094 raft_consensus.cc:359] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.970446  7094 raft_consensus.cc:385] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:47.970846  7094 raft_consensus.cc:740] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:47.971724  7094 consensus_queue.cc:260] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:47.979028  7097 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:47.983834  7094 ts_tablet_manager.cc:1434] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.002s
I20260709 12:27:47.990199  7096 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:47.990811  7096 ts_tablet_manager.cc:1403] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.084s	user 0.027s	sys 0.035s
I20260709 12:27:47.994655  7094 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:47.998567  7097 tablet_bootstrap.cc:654] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:47.996251  7095 tablet_bootstrap.cc:492] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:47.999155  7095 ts_tablet_manager.cc:1403] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.090s	user 0.017s	sys 0.000s
I20260709 12:27:48.000370  7312 consensus_queue.cc:1048] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.004377  7096 raft_consensus.cc:359] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.005059  7096 raft_consensus.cc:385] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.005333  7096 raft_consensus.cc:740] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.006007  7096 consensus_queue.cc:260] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.011637  7096 ts_tablet_manager.cc:1434] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.002s
I20260709 12:27:48.012619  7096 tablet_bootstrap.cc:492] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:48.013116  7323 raft_consensus.cc:493] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.013614  7323 raft_consensus.cc:515] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.015794  7323 leader_election.cc:290] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:48.019057  7096 tablet_bootstrap.cc:654] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.021090  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:48.019889  7095 raft_consensus.cc:359] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.022420  7095 raft_consensus.cc:385] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.022785  7095 raft_consensus.cc:740] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.024094  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:48.025022  6827 raft_consensus.cc:2468] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:48.031692  7095 consensus_queue.cc:260] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.033380  7094 tablet_bootstrap.cc:654] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:48.037974  7016 leader_election.cc:343] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.038980  7111 consensus_queue.cc:1048] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.045312  7013 leader_election.cc:304] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:48.047444  7140 raft_consensus.cc:2804] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.047816  7140 raft_consensus.cc:493] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.048128  7140 raft_consensus.cc:3060] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.053661  7095 ts_tablet_manager.cc:1434] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.001s
I20260709 12:27:48.060366  7095 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:48.066496  7205 consensus_queue.cc:1048] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.068662  7097 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:48.069234  7097 ts_tablet_manager.cc:1403] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.095s	user 0.038s	sys 0.035s
I20260709 12:27:48.061931  7140 raft_consensus.cc:515] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.072654  7094 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:48.072566  7097 raft_consensus.cc:359] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.073180  7094 ts_tablet_manager.cc:1403] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.079s	user 0.004s	sys 0.008s
I20260709 12:27:48.073338  7097 raft_consensus.cc:385] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.073509  7140 leader_election.cc:290] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:48.073753  7097 raft_consensus.cc:740] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.074579  7097 consensus_queue.cc:260] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.075865  7094 raft_consensus.cc:359] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.076519  7094 raft_consensus.cc:385] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.076642  7097 ts_tablet_manager.cc:1434] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:27:48.076830  7094 raft_consensus.cc:740] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.077857  7097 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:48.079239  7095 tablet_bootstrap.cc:654] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.080781  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:48.081977  7094 consensus_queue.cc:260] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.084292  7097 tablet_bootstrap.cc:654] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.084476  7094 ts_tablet_manager.cc:1434] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:27:48.090812  7096 tablet_bootstrap.cc:492] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:48.091280  7096 ts_tablet_manager.cc:1403] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.022s
I20260709 12:27:48.093926  7096 raft_consensus.cc:359] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.094563  7096 raft_consensus.cc:385] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.094846  7096 raft_consensus.cc:740] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.095484  7096 consensus_queue.cc:260] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.096138  7097 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:48.096645  7097 ts_tablet_manager.cc:1403] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 12:27:48.097936  7096 ts_tablet_manager.cc:1434] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
W20260709 12:27:48.099212  7016 leader_election.cc:343] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.099385  7097 raft_consensus.cc:359] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.100304  7097 raft_consensus.cc:385] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.100616  7097 raft_consensus.cc:740] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.101401  7097 consensus_queue.cc:260] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.103123  7097 ts_tablet_manager.cc:1434] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 12:27:48.104044  7097 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:48.109737  7097 tablet_bootstrap.cc:654] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.109841  7094 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:48.116601  7095 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:48.117163  7095 ts_tablet_manager.cc:1403] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.000s
I20260709 12:27:48.118759  7222 raft_consensus.cc:493] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.119309  7222 raft_consensus.cc:515] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.119735  7096 tablet_bootstrap.cc:492] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:48.126197  7222 leader_election.cc:290] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
W20260709 12:27:48.117578  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 60345.7us
I20260709 12:27:48.132944  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:48.133440  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0698605f5af41fcbc332b4dc8857606" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:48.133554  6824 raft_consensus.cc:3060] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:48.134924  6865 leader_election.cc:343] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.138736  6827 raft_consensus.cc:3060] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.140843  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0698605f5af41fcbc332b4dc8857606" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:48.141531  7048 raft_consensus.cc:2468] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:48.144965  7263 raft_consensus.cc:493] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.145603  7263 raft_consensus.cc:515] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.146436  6866 leader_election.cc:304] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:48.148263  7263 leader_election.cc:290] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:48.148725  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c78f0ec1af4414a7e52c69f19b6c27" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:48.149286  7222 raft_consensus.cc:2804] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.149710  7222 raft_consensus.cc:493] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.150084  7222 raft_consensus.cc:3060] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.150691  7095 raft_consensus.cc:359] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.151638  7095 raft_consensus.cc:385] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.151993  7095 raft_consensus.cc:740] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.153079  7323 consensus_queue.cc:1048] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.152935  7095 consensus_queue.cc:260] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.166890  7095 ts_tablet_manager.cc:1434] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.001s
I20260709 12:27:48.168661  7095 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:48.169320  7097 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:48.169909  7097 ts_tablet_manager.cc:1403] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.039s
I20260709 12:27:48.171129  7094 tablet_bootstrap.cc:654] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.178017  7095 tablet_bootstrap.cc:654] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.182113  7277 raft_consensus.cc:493] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.182400  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c78f0ec1af4414a7e52c69f19b6c27" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:48.182732  7277 raft_consensus.cc:515] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.190412  7096 tablet_bootstrap.cc:654] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.192786  6791 leader_election.cc:304] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:48.193783  7321 raft_consensus.cc:2749] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:48.194650  7140 consensus_queue.cc:1048] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.184141  7097 raft_consensus.cc:359] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.198663  7097 raft_consensus.cc:385] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.199034  7097 raft_consensus.cc:740] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.199241  6824 raft_consensus.cc:2468] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:48.199909  7097 consensus_queue.cc:260] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.207778  7013 leader_election.cc:304] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:48.208616  7323 raft_consensus.cc:2804] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.209084  7323 raft_consensus.cc:697] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:48.211635  7323 consensus_queue.cc:237] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.213608  7097 ts_tablet_manager.cc:1434] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 12:27:48.219493  7094 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:48.220093  7094 ts_tablet_manager.cc:1403] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.111s	user 0.009s	sys 0.008s
I20260709 12:27:48.220450  7097 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:48.221809  7096 tablet_bootstrap.cc:492] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:48.222318  7096 ts_tablet_manager.cc:1403] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.014s
I20260709 12:27:48.223124  7094 raft_consensus.cc:359] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.223846  7094 raft_consensus.cc:385] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.224154  7094 raft_consensus.cc:740] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.225342  7094 consensus_queue.cc:260] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.225680  7096 raft_consensus.cc:359] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.210999  7222 raft_consensus.cc:515] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.229835  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d0e6da2d6d4fa6abb2757dff7de498" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:48.231348  7096 raft_consensus.cc:385] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.232012  7096 raft_consensus.cc:740] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.236460  7050 raft_consensus.cc:2468] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:48.237404  7094 ts_tablet_manager.cc:1434] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:27:48.237947  7097 tablet_bootstrap.cc:654] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.240042  7094 tablet_bootstrap.cc:492] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:48.244261  7322 consensus_queue.cc:1048] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:48.246860  7277 leader_election.cc:290] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:48.249500  7096 consensus_queue.cc:260] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.271512  6825 raft_consensus.cc:3060] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.285166  7096 ts_tablet_manager.cc:1434] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.002s
I20260709 12:27:48.261888  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0698605f5af41fcbc332b4dc8857606" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:48.252789  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d0e6da2d6d4fa6abb2757dff7de498" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:48.253254  6866 leader_election.cc:304] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:48.287989  7096 tablet_bootstrap.cc:492] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:48.282356  7154 raft_consensus.cc:493] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.282692  7222 leader_election.cc:290] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:48.295734  7356 raft_consensus.cc:2804] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.296209  7356 raft_consensus.cc:493] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.296563  7356 raft_consensus.cc:3060] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:48.290139  6865 leader_election.cc:343] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.301015  7318 consensus_queue.cc:1048] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.303342  6724 catalog_manager.cc:5697] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:48.295130  6827 raft_consensus.cc:2468] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:48.307238  7222 raft_consensus.cc:493] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.307714  7222 raft_consensus.cc:515] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.309270  7095 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:48.309691  7222 leader_election.cc:290] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:48.309795  7095 ts_tablet_manager.cc:1403] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.027s
I20260709 12:27:48.316661  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:48.318158  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:48.319980  6865 leader_election.cc:343] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.288415  7154 raft_consensus.cc:515] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.329018  6973 raft_consensus.cc:2468] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:48.317605  7095 raft_consensus.cc:359] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.331696  7095 raft_consensus.cc:385] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.332091  7095 raft_consensus.cc:740] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.337292  7095 consensus_queue.cc:260] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.335777  6865 leader_election.cc:304] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:48.346659  7094 tablet_bootstrap.cc:654] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.349871  7263 raft_consensus.cc:493] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.353340  7359 raft_consensus.cc:2804] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.353772  7359 raft_consensus.cc:493] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.354097  7359 raft_consensus.cc:3060] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.350579  7263 raft_consensus.cc:515] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.355567  7356 raft_consensus.cc:515] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.359006  7263 leader_election.cc:290] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:48.358929  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:48.360297  6973 raft_consensus.cc:2468] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:48.358929  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:48.361198  7359 raft_consensus.cc:515] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.361907  6790 leader_election.cc:304] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
W20260709 12:27:48.362207  6939 leader_election.cc:343] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.362622  7095 ts_tablet_manager.cc:1434] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.048s	user 0.005s	sys 0.002s
I20260709 12:27:48.362906  7263 raft_consensus.cc:2804] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.363274  7263 raft_consensus.cc:493] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.363617  7263 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.363675  7095 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:48.365221  7097 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:48.365845  7097 ts_tablet_manager.cc:1403] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.146s	user 0.009s	sys 0.030s
I20260709 12:27:48.369833  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:48.371856  7315 raft_consensus.cc:493] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.370642  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:48.372530  7315 raft_consensus.cc:515] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.375089  7154 leader_election.cc:290] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
W20260709 12:27:48.375840  6865 leader_election.cc:343] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.377861  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:48.378559  7048 raft_consensus.cc:2468] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:48.379495  7315 leader_election.cc:290] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:48.380563  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:48.380687  6940 leader_election.cc:304] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
W20260709 12:27:48.382036  6939 leader_election.cc:343] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.385601  7263 raft_consensus.cc:515] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.364187  7359 leader_election.cc:290] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:48.380242  6973 raft_consensus.cc:3060] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.388208  7315 raft_consensus.cc:2804] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.389528  7356 leader_election.cc:290] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:48.390283  7049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0698605f5af41fcbc332b4dc8857606" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:48.394402  7049 raft_consensus.cc:3060] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.394829  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d0e6da2d6d4fa6abb2757dff7de498" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:48.395419  6826 raft_consensus.cc:3060] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.398871  7095 tablet_bootstrap.cc:654] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.392166  7315 raft_consensus.cc:493] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.404767  6826 raft_consensus.cc:2468] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:48.405083  7315 raft_consensus.cc:3060] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.406034  6865 leader_election.cc:304] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:48.406950  7359 raft_consensus.cc:2804] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.407428  7359 raft_consensus.cc:697] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:48.408233  7359 consensus_queue.cc:237] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.414922  7094 tablet_bootstrap.cc:492] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:48.415453  7094 ts_tablet_manager.cc:1403] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.177s	user 0.017s	sys 0.017s
I20260709 12:27:48.416683  7097 raft_consensus.cc:359] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.417363  7097 raft_consensus.cc:385] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.417619  7097 raft_consensus.cc:740] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.419838  7094 raft_consensus.cc:359] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.420549  7094 raft_consensus.cc:385] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.420806  7094 raft_consensus.cc:740] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.421437  7094 consensus_queue.cc:260] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.423805  7094 ts_tablet_manager.cc:1434] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:27:48.412811  7315 raft_consensus.cc:515] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.424121  7318 consensus_queue.cc:1048] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.424351  7097 consensus_queue.cc:260] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.435564  6973 raft_consensus.cc:2468] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:48.436823  6865 leader_election.cc:304] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:48.438288  7356 raft_consensus.cc:2804] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.439000  7356 raft_consensus.cc:697] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:48.440111  7047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:48.441817  7315 leader_election.cc:290] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:48.444891  7097 ts_tablet_manager.cc:1434] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.079s	user 0.006s	sys 0.002s
I20260709 12:27:48.446946  7263 leader_election.cc:290] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:48.447769  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:48.449241  7094 tablet_bootstrap.cc:492] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:48.449105  7321 consensus_queue.cc:1048] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.449553  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:48.451146  7277 raft_consensus.cc:493] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.462718  7277 raft_consensus.cc:515] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.449926  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:48.463722  6975 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.464648  7277 leader_election.cc:290] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:48.462711  7047 raft_consensus.cc:3060] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.465737  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:48.466615  7356 consensus_queue.cc:237] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
W20260709 12:27:48.467362  6865 leader_election.cc:343] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.457365  7095 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:48.470098  7095 ts_tablet_manager.cc:1403] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.107s	user 0.014s	sys 0.001s
W20260709 12:27:48.471885  6939 leader_election.cc:343] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.474167  7047 raft_consensus.cc:2468] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:48.476034  6975 raft_consensus.cc:2468] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:48.478461  7097 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:48.481534  6790 leader_election.cc:304] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:48.482635  7205 raft_consensus.cc:2804] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.483080  7205 raft_consensus.cc:697] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:48.483915  7205 consensus_queue.cc:237] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.470538  7049 raft_consensus.cc:2468] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:48.489373  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:48.490263  6901 raft_consensus.cc:2468] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:48.499490  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:48.500262  6901 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.498937  6940 leader_election.cc:304] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:48.514274  7318 raft_consensus.cc:2804] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.514810  7318 raft_consensus.cc:697] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:48.519117  7110 consensus_queue.cc:1048] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:27:48.519269  7315 consensus_queue.cc:1048] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:27:48.521215  7045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d0e6da2d6d4fa6abb2757dff7de498" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:48.530479  7045 raft_consensus.cc:3060] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.524462  7095 raft_consensus.cc:359] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.533816  7095 raft_consensus.cc:385] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.534072  7095 raft_consensus.cc:740] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.536263  6901 raft_consensus.cc:2468] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:48.537514  7095 consensus_queue.cc:260] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
W20260709 12:27:48.515703  7362 log.cc:927] Time spent T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:27:48.518936  6942 leader_election.cc:304] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:48.515709  7318 consensus_queue.cc:237] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.548456  7110 raft_consensus.cc:2749] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:48.556656  7046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:48.557101  6724 catalog_manager.cc:5697] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:48.565919  7095 ts_tablet_manager.cc:1434] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.096s	user 0.007s	sys 0.000s
I20260709 12:27:48.581084  7095 tablet_bootstrap.cc:492] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:48.575500  7321 consensus_queue.cc:1048] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:27:48.596771  7094 tablet_bootstrap.cc:654] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.604789  6866 leader_election.cc:304] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:48.606379  7277 raft_consensus.cc:2804] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.606900  7277 raft_consensus.cc:697] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:48.607260  7096 tablet_bootstrap.cc:654] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.607770  7277 consensus_queue.cc:237] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.610857  7045 raft_consensus.cc:2468] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:48.613507  6721 catalog_manager.cc:5697] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:48.606395  6866 leader_election.cc:343] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.614915  6866 leader_election.cc:304] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:48.615693  7356 raft_consensus.cc:2749] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:48.620167  7097 tablet_bootstrap.cc:654] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:48.630534  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:48.655807  7095 tablet_bootstrap.cc:654] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.657945  6724 catalog_manager.cc:5697] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:48.661597  6724 catalog_manager.cc:5697] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:48.663183  7096 tablet_bootstrap.cc:492] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:48.663457  7094 tablet_bootstrap.cc:492] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:48.663703  7096 ts_tablet_manager.cc:1403] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.376s	user 0.011s	sys 0.006s
I20260709 12:27:48.663944  7094 ts_tablet_manager.cc:1403] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.215s	user 0.017s	sys 0.008s
I20260709 12:27:48.666587  7140 consensus_queue.cc:1048] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.666860  7094 raft_consensus.cc:359] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.667601  7094 raft_consensus.cc:385] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.668145  7094 raft_consensus.cc:740] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.668812  7094 consensus_queue.cc:260] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.670967  7094 ts_tablet_manager.cc:1434] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:27:48.671917  7094 tablet_bootstrap.cc:492] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:48.672750  7096 raft_consensus.cc:359] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.673750  7096 raft_consensus.cc:385] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.674379  7096 raft_consensus.cc:740] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.677322  7096 consensus_queue.cc:260] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.687829  7094 tablet_bootstrap.cc:654] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.694788  7096 ts_tablet_manager.cc:1434] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:27:48.695956  7096 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:48.720243  7096 tablet_bootstrap.cc:654] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.722138  7315 raft_consensus.cc:493] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.726243  7315 raft_consensus.cc:515] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.727033  7097 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:48.727556  7097 ts_tablet_manager.cc:1403] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.249s	user 0.018s	sys 0.012s
I20260709 12:27:48.728426  7315 leader_election.cc:290] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:48.730639  7097 raft_consensus.cc:359] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.731268  7097 raft_consensus.cc:385] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.731494  7097 raft_consensus.cc:740] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.731855  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0698605f5af41fcbc332b4dc8857606" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:48.734486  7140 raft_consensus.cc:493] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.735046  7140 raft_consensus.cc:515] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.737524  7140 leader_election.cc:290] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:48.737648  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0698605f5af41fcbc332b4dc8857606" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:48.738551  7050 raft_consensus.cc:2393] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 1: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:48.732510  7097 consensus_queue.cc:260] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.739667  6940 leader_election.cc:304] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:48.740520  7315 raft_consensus.cc:3060] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.739677  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d766dbe21d42a0ab36d005d89289c9" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:48.740983  7097 ts_tablet_manager.cc:1434] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:27:48.742069  7097 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
W20260709 12:27:48.742336  7013 leader_election.cc:343] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:48.754395  7016 consensus_peers.cc:597] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:48.762910  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d766dbe21d42a0ab36d005d89289c9" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:48.766916  7016 leader_election.cc:343] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.767380  7016 leader_election.cc:304] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:48.768926  7323 raft_consensus.cc:2749] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:48.769886  7154 raft_consensus.cc:493] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.769613  6723 catalog_manager.cc:5697] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:48.770390  7154 raft_consensus.cc:515] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.775393  7315 raft_consensus.cc:2749] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:48.776839  7154 leader_election.cc:290] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:48.782076  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:48.785504  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:48.787878  7050 raft_consensus.cc:2468] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
W20260709 12:27:48.790599  6942 leader_election.cc:343] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.792412  6940 leader_election.cc:304] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:48.793793  7097 tablet_bootstrap.cc:654] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.794124  7318 raft_consensus.cc:2804] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.794591  7318 raft_consensus.cc:493] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.794951  7318 raft_consensus.cc:3060] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.811151  7095 tablet_bootstrap.cc:492] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:48.811676  7095 ts_tablet_manager.cc:1403] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.235s	user 0.013s	sys 0.012s
I20260709 12:27:48.814204  7095 raft_consensus.cc:359] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.816506  7095 raft_consensus.cc:385] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.816807  7095 raft_consensus.cc:740] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.817487  7095 consensus_queue.cc:260] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.819409  7096 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:48.819559  7095 ts_tablet_manager.cc:1434] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 12:27:48.819788  7094 tablet_bootstrap.cc:492] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:48.820154  7096 ts_tablet_manager.cc:1403] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.124s	user 0.012s	sys 0.008s
I20260709 12:27:48.821048  7095 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:48.821496  7318 raft_consensus.cc:515] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.825865  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:48.826457  7050 raft_consensus.cc:3060] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.821722  7094 ts_tablet_manager.cc:1403] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.150s	user 0.018s	sys 0.028s
I20260709 12:27:48.830027  7094 raft_consensus.cc:359] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.830649  7094 raft_consensus.cc:385] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.830930  7094 raft_consensus.cc:740] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.831550  7094 consensus_queue.cc:260] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.830677  7096 raft_consensus.cc:359] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.832729  7096 raft_consensus.cc:385] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.832993  7096 raft_consensus.cc:740] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.833562  7277 raft_consensus.cc:493] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.833954  7094 ts_tablet_manager.cc:1434] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:27:48.834491  7095 tablet_bootstrap.cc:654] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.834102  7277 raft_consensus.cc:515] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.837038  6825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2fcd7006bc840a0bb5ea27365793f86" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:48.837430  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2fcd7006bc840a0bb5ea27365793f86" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:48.837710  6825 raft_consensus.cc:2468] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:48.838907  7277 leader_election.cc:290] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
W20260709 12:27:48.838971  6865 leader_election.cc:343] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.841706  7050 raft_consensus.cc:2468] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:48.837565  7094 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:48.842540  7096 consensus_queue.cc:260] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.845599  7318 leader_election.cc:290] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:48.853803  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:48.849627  7277 consensus_queue.cc:1048] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:27:48.854439  6865 leader_election.cc:304] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:48.855520  7359 raft_consensus.cc:2804] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.855878  7359 raft_consensus.cc:493] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.856221  7359 raft_consensus.cc:3060] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.861351  7097 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:48.861517  7094 tablet_bootstrap.cc:654] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.861882  7097 ts_tablet_manager.cc:1403] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.120s	user 0.014s	sys 0.011s
I20260709 12:27:48.865299  6940 leader_election.cc:304] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:48.868567  7097 raft_consensus.cc:359] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.869268  7097 raft_consensus.cc:385] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.869553  7097 raft_consensus.cc:740] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.870508  7110 raft_consensus.cc:2804] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.871260  7097 consensus_queue.cc:260] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.872383  7321 raft_consensus.cc:493] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.872857  7321 raft_consensus.cc:515] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.873624  7359 raft_consensus.cc:515] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.875054  7321 leader_election.cc:290] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:48.879629  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:48.880283  6973 raft_consensus.cc:2468] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:48.880995  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2fcd7006bc840a0bb5ea27365793f86" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:48.881596  6826 raft_consensus.cc:3060] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.881589  6790 leader_election.cc:304] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:48.882110  7110 raft_consensus.cc:697] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:48.856181  7096 ts_tablet_manager.cc:1434] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.001s
W20260709 12:27:48.884974  6942 leader_election.cc:343] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.886260  7096 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:48.890429  7321 raft_consensus.cc:2804] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:48.891090  7321 raft_consensus.cc:493] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.891652  7321 raft_consensus.cc:3060] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.892678  6826 raft_consensus.cc:2468] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:48.893867  7359 leader_election.cc:290] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:48.895308  7096 tablet_bootstrap.cc:654] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.895819  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:48.896505  7048 raft_consensus.cc:2468] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:48.896790  6865 leader_election.cc:304] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:48.897212  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2fcd7006bc840a0bb5ea27365793f86" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
W20260709 12:27:48.898497  6865 leader_election.cc:343] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:48.898569  7277 raft_consensus.cc:2804] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.899118  7277 raft_consensus.cc:697] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:48.899966  7277 consensus_queue.cc:237] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.890733  7110 consensus_queue.cc:237] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:48.917987  7321 raft_consensus.cc:515] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.915328  7097 ts_tablet_manager.cc:1434] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 12:27:48.920055  7097 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:48.920898  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:48.921506  6973 raft_consensus.cc:3060] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.928676  6973 raft_consensus.cc:2468] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:48.929785  6790 leader_election.cc:304] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:48.930712  7205 raft_consensus.cc:2804] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:48.931207  7205 raft_consensus.cc:697] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:48.932004  7205 consensus_queue.cc:237] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.941555  7094 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:48.942775  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:48.943465  7050 raft_consensus.cc:3060] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.949330  7097 tablet_bootstrap.cc:654] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.942909  7321 leader_election.cc:290] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:48.951522  7050 raft_consensus.cc:2468] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:48.953598  7095 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:48.954175  7095 ts_tablet_manager.cc:1403] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.133s	user 0.022s	sys 0.025s
I20260709 12:27:48.955897  7094 ts_tablet_manager.cc:1403] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.009s
I20260709 12:27:48.956123  7096 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:48.956629  7096 ts_tablet_manager.cc:1403] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.001s
I20260709 12:27:48.964285  7222 consensus_queue.cc:1048] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.966861  7096 raft_consensus.cc:359] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.967501  7096 raft_consensus.cc:385] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.967756  7096 raft_consensus.cc:740] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.969786  7097 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:48.969532  7094 raft_consensus.cc:359] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.970234  7094 raft_consensus.cc:385] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.970310  7097 ts_tablet_manager.cc:1403] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.000s
I20260709 12:27:48.970542  7094 raft_consensus.cc:740] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.971019  7096 consensus_queue.cc:260] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.971280  7094 consensus_queue.cc:260] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:48.973484  7096 ts_tablet_manager.cc:1434] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:27:48.974502  7096 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:48.975039  7097 raft_consensus.cc:359] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.973484  7094 ts_tablet_manager.cc:1434] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 12:27:48.975879  7097 raft_consensus.cc:385] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.976336  7097 raft_consensus.cc:740] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.976835  7094 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:48.977142  7097 consensus_queue.cc:260] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:48.979553  7095 raft_consensus.cc:359] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.980206  7095 raft_consensus.cc:385] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:48.980327  6825 raft_consensus.cc:3060] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:48.980521  7095 raft_consensus.cc:740] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:48.985708  7095 consensus_queue.cc:260] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:48.987290  7096 tablet_bootstrap.cc:654] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:48.989419  7318 consensus_queue.cc:1048] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:48.991552  6725 catalog_manager.cc:5697] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:48.988169  6722 catalog_manager.cc:5697] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:48.998871  7095 ts_tablet_manager.cc:1434] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 12:27:48.998265  6721 catalog_manager.cc:5697] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:48.999938  7095 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:49.002610  7097 ts_tablet_manager.cc:1434] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.002s
I20260709 12:27:49.003293  7094 tablet_bootstrap.cc:654] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.003633  7097 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:49.010994  7096 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:49.011515  7096 ts_tablet_manager.cc:1403] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.002s
I20260709 12:27:49.012290  7097 tablet_bootstrap.cc:654] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.014151  7096 raft_consensus.cc:359] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.014879  7096 raft_consensus.cc:385] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.015136  7096 raft_consensus.cc:740] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.015815  7096 consensus_queue.cc:260] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.017935  7096 ts_tablet_manager.cc:1434] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:27:49.031174  7222 consensus_queue.cc:1048] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.032394  7095 tablet_bootstrap.cc:654] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.036319  7318 raft_consensus.cc:493] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.036837  7318 raft_consensus.cc:515] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.037276  7096 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:49.038995  7318 leader_election.cc:290] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:49.041066  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2fcd7006bc840a0bb5ea27365793f86" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:49.042790  6865 consensus_peers.cc:597] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:49.046288  7097 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:49.047273  7096 tablet_bootstrap.cc:654] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.046862  7097 ts_tablet_manager.cc:1403] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.001s
I20260709 12:27:49.052137  7097 raft_consensus.cc:359] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.052779  7097 raft_consensus.cc:385] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.053020  7097 raft_consensus.cc:740] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.064349  7096 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:49.065091  7097 consensus_queue.cc:260] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.065024  7096 ts_tablet_manager.cc:1403] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.005s
I20260709 12:27:49.070961  7096 raft_consensus.cc:359] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.071898  7096 raft_consensus.cc:385] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.072221  7096 raft_consensus.cc:740] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.072431  7097 ts_tablet_manager.cc:1434] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 12:27:49.073472  7097 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:49.073076  7096 consensus_queue.cc:260] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.078280  7315 consensus_queue.cc:1048] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.082858  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2fcd7006bc840a0bb5ea27365793f86" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.084519  7096 ts_tablet_manager.cc:1434] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:27:49.084936  6826 raft_consensus.cc:2393] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 1: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:49.085541  7096 tablet_bootstrap.cc:492] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:49.088405  6939 leader_election.cc:304] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:49.089154  7110 raft_consensus.cc:3060] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.097157  7263 consensus_queue.cc:1048] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.097836  7096 tablet_bootstrap.cc:654] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.102180  7321 consensus_queue.cc:1048] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.105981  7097 tablet_bootstrap.cc:654] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.109375  7096 tablet_bootstrap.cc:492] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:49.109836  7096 ts_tablet_manager.cc:1403] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.010s
I20260709 12:27:49.110740  7095 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:49.111186  7095 ts_tablet_manager.cc:1403] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.111s	user 0.020s	sys 0.036s
I20260709 12:27:49.112442  7096 raft_consensus.cc:359] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.113129  7096 raft_consensus.cc:385] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.113372  7096 raft_consensus.cc:740] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.114082  7095 raft_consensus.cc:359] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.114365  7096 consensus_queue.cc:260] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.116457  7096 ts_tablet_manager.cc:1434] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:27:49.117416  7096 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:49.119783  7356 consensus_queue.cc:1048] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.122962  7096 tablet_bootstrap.cc:654] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.114787  7095 raft_consensus.cc:385] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.125638  7095 raft_consensus.cc:740] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.126349  7095 consensus_queue.cc:260] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.128494  7095 ts_tablet_manager.cc:1434] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:27:49.129441  7095 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:49.131224  7359 raft_consensus.cc:493] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.131726  7359 raft_consensus.cc:515] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.133824  7359 leader_election.cc:290] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.135346  7095 tablet_bootstrap.cc:654] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.135260  6825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.137284  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.138810  6866 leader_election.cc:304] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:49.140152  7359 raft_consensus.cc:2749] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.148041  7095 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:49.148658  7095 ts_tablet_manager.cc:1403] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.007s
I20260709 12:27:49.157606  7095 raft_consensus.cc:359] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.158274  7095 raft_consensus.cc:385] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.158548  7095 raft_consensus.cc:740] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.159129  7095 consensus_queue.cc:260] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.166308  7315 raft_consensus.cc:493] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.167061  7315 raft_consensus.cc:515] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.168687  7095 ts_tablet_manager.cc:1434] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:27:49.169718  7095 tablet_bootstrap.cc:492] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:49.171356  7315 leader_election.cc:290] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:49.203634  7094 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:49.204159  7094 ts_tablet_manager.cc:1403] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.228s	user 0.019s	sys 0.024s
I20260709 12:27:49.206960  7094 raft_consensus.cc:359] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.207603  7094 raft_consensus.cc:385] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.207867  7094 raft_consensus.cc:740] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.208549  7094 consensus_queue.cc:260] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.173003  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d766dbe21d42a0ab36d005d89289c9" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:49.210682  7094 ts_tablet_manager.cc:1434] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:27:49.211624  7094 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:49.216035  6900 raft_consensus.cc:2468] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:49.175429  7095 tablet_bootstrap.cc:654] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.218451  6942 leader_election.cc:304] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:49.220141  7315 raft_consensus.cc:2804] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.220505  7315 raft_consensus.cc:493] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.220870  7315 raft_consensus.cc:3060] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.227602  7097 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:49.228405  7097 ts_tablet_manager.cc:1403] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.155s	user 0.024s	sys 0.018s
I20260709 12:27:49.230852  7315 raft_consensus.cc:515] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.232267  7097 raft_consensus.cc:359] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.233239  7097 raft_consensus.cc:385] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.233505  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d766dbe21d42a0ab36d005d89289c9" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:49.233585  7097 raft_consensus.cc:740] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.234197  6900 raft_consensus.cc:3060] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.234697  7097 consensus_queue.cc:260] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.238266  7097 ts_tablet_manager.cc:1434] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:27:49.239780  7097 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:49.241483  6900 raft_consensus.cc:2468] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:49.242817  6942 leader_election.cc:304] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:49.243696  7154 raft_consensus.cc:2804] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:49.180610  7110 raft_consensus.cc:2749] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.244232  7154 raft_consensus.cc:697] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:49.188191  7096 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:49.246031  7096 ts_tablet_manager.cc:1403] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.129s	user 0.029s	sys 0.011s
I20260709 12:27:49.247754  7097 tablet_bootstrap.cc:654] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.194355  7222 consensus_queue.cc:1048] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.174538  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d766dbe21d42a0ab36d005d89289c9" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.250206  7050 raft_consensus.cc:2468] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:49.217244  7094 tablet_bootstrap.cc:654] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.245092  7154 consensus_queue.cc:237] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.248840  7096 raft_consensus.cc:359] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.257329  7096 raft_consensus.cc:385] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.258507  7096 raft_consensus.cc:740] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.259316  7096 consensus_queue.cc:260] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.262023  7096 ts_tablet_manager.cc:1434] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:27:49.263110  7096 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:49.265749  7318 raft_consensus.cc:493] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.266275  7318 raft_consensus.cc:515] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.268433  7318 leader_election.cc:290] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.269287  7097 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:49.269410  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb45dbc587b34cb7843e48180e0219c3" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.269841  7097 ts_tablet_manager.cc:1403] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.012s
I20260709 12:27:49.270195  6824 raft_consensus.cc:2468] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:49.272538  6939 leader_election.cc:304] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:49.272745  7097 raft_consensus.cc:359] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.273447  7097 raft_consensus.cc:385] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.273720  7097 raft_consensus.cc:740] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.274690  7097 consensus_queue.cc:260] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.273851  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb45dbc587b34cb7843e48180e0219c3" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.275756  7048 raft_consensus.cc:2468] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:49.276666  7097 ts_tablet_manager.cc:1434] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:27:49.275243  7318 raft_consensus.cc:2804] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.277451  7318 raft_consensus.cc:493] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.277679  7097 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:49.277895  7318 raft_consensus.cc:3060] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.304224  7096 tablet_bootstrap.cc:654] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.306236  7094 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:49.306404  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2d766dbe21d42a0ab36d005d89289c9" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:49.306743  7094 ts_tablet_manager.cc:1403] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.095s	user 0.004s	sys 0.009s
I20260709 12:27:49.307245  7048 raft_consensus.cc:3060] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.308509  7095 tablet_bootstrap.cc:492] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:49.308974  7095 ts_tablet_manager.cc:1403] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.139s	user 0.017s	sys 0.036s
I20260709 12:27:49.309634  7094 raft_consensus.cc:359] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.310339  7094 raft_consensus.cc:385] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.310607  7094 raft_consensus.cc:740] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.311228  7094 consensus_queue.cc:260] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.311573  7096 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:49.312204  7096 ts_tablet_manager.cc:1403] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.002s
I20260709 12:27:49.311635  7095 raft_consensus.cc:359] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.314388  7094 ts_tablet_manager.cc:1434] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:27:49.314627  7095 raft_consensus.cc:385] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.314914  7095 raft_consensus.cc:740] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.314991  7322 raft_consensus.cc:493] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.315299  7094 tablet_bootstrap.cc:492] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
W20260709 12:27:49.315075  6942 consensus_peers.cc:597] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:49.315644  7322 raft_consensus.cc:515] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.317762  7322 leader_election.cc:290] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.316664  7096 raft_consensus.cc:359] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.318429  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fa5eb82f684b449d82176587520e4d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:49.318426  7315 leader_election.cc:290] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:49.320014  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fa5eb82f684b449d82176587520e4d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.306851  7097 tablet_bootstrap.cc:654] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:49.322507  7013 leader_election.cc:343] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:49.323364  7013 leader_election.cc:343] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.323764  7013 leader_election.cc:304] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:49.324549  7322 raft_consensus.cc:2749] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.325047  7096 raft_consensus.cc:385] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.325350  7096 raft_consensus.cc:740] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.315610  7095 consensus_queue.cc:260] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.303921  7318 raft_consensus.cc:515] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.328142  7095 ts_tablet_manager.cc:1434] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:27:49.329118  7095 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:49.329404  7094 tablet_bootstrap.cc:654] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.325935  7096 consensus_queue.cc:260] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.332149  7096 ts_tablet_manager.cc:1434] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:27:49.333456  7096 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:49.335263  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb45dbc587b34cb7843e48180e0219c3" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:49.335877  6824 raft_consensus.cc:3060] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.339262  7318 leader_election.cc:290] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.341753  6721 catalog_manager.cc:5697] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:49.343570  7097 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:49.344311  7097 ts_tablet_manager.cc:1403] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.000s
I20260709 12:27:49.346179  7322 raft_consensus.cc:493] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.346668  7322 raft_consensus.cc:515] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.347790  7263 consensus_queue.cc:1048] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.348600  7322 leader_election.cc:290] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:49.349862  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb45dbc587b34cb7843e48180e0219c3" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:49.350169  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfa9e71770a4e3dac4348d84a9cc2d6" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:49.350445  7050 raft_consensus.cc:3060] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.350553  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfa9e71770a4e3dac4348d84a9cc2d6" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:49.352186  7016 leader_election.cc:343] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.324301  7048 raft_consensus.cc:2468] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
W20260709 12:27:49.352958  7013 leader_election.cc:343] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.353431  7013 leader_election.cc:304] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:49.354669  7322 raft_consensus.cc:2749] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.357183  7050 raft_consensus.cc:2468] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:49.359025  7096 tablet_bootstrap.cc:654] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.359788  7094 tablet_bootstrap.cc:492] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:49.360321  7094 ts_tablet_manager.cc:1403] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.007s
I20260709 12:27:49.361079  6940 leader_election.cc:304] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:49.361867  7318 raft_consensus.cc:2804] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:49.362340  7318 raft_consensus.cc:697] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:49.363149  7318 consensus_queue.cc:237] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.352236  6824 raft_consensus.cc:2468] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:49.365334  6900 raft_consensus.cc:3060] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.366442  7095 tablet_bootstrap.cc:654] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.366880  7263 raft_consensus.cc:493] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.367393  7263 raft_consensus.cc:515] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.371150  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:49.371757  6901 raft_consensus.cc:2468] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:49.372881  6793 leader_election.cc:304] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:49.373697  7321 raft_consensus.cc:2804] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.374037  7321 raft_consensus.cc:493] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.374379  7321 raft_consensus.cc:3060] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.375242  7097 raft_consensus.cc:359] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.376137  7097 raft_consensus.cc:385] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.376461  7097 raft_consensus.cc:740] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.377297  7097 consensus_queue.cc:260] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.380231  7094 raft_consensus.cc:359] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.380909  7094 raft_consensus.cc:385] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.381166  7094 raft_consensus.cc:740] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.382472  7095 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:49.383002  7095 ts_tablet_manager.cc:1403] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.054s	user 0.006s	sys 0.006s
I20260709 12:27:49.383549  7096 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:49.384171  7096 ts_tablet_manager.cc:1403] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.000s
I20260709 12:27:49.386060  7097 ts_tablet_manager.cc:1434] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:27:49.387439  7095 raft_consensus.cc:359] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.393847  7263 leader_election.cc:290] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:49.394145  7095 raft_consensus.cc:385] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.394631  7095 raft_consensus.cc:740] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.392087  7096 raft_consensus.cc:359] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.397130  7096 raft_consensus.cc:385] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.397418  7096 raft_consensus.cc:740] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.397502  7095 consensus_queue.cc:260] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.398044  7096 consensus_queue.cc:260] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.381824  7094 consensus_queue.cc:260] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.400233  7095 ts_tablet_manager.cc:1434] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.002s
I20260709 12:27:49.400532  7094 ts_tablet_manager.cc:1434] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 12:27:49.401286  7095 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:49.401527  7094 tablet_bootstrap.cc:492] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:49.402377  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.402666  7205 raft_consensus.cc:493] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.403055  7048 raft_consensus.cc:2468] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:49.403239  7205 raft_consensus.cc:515] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.382117  7321 raft_consensus.cc:515] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.407917  7321 leader_election.cc:290] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:49.408272  7096 ts_tablet_manager.cc:1434] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:27:49.408442  7095 tablet_bootstrap.cc:654] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.410391  7323 raft_consensus.cc:493] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.410580  7205 leader_election.cc:290] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:49.410933  7323 raft_consensus.cc:515] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.411199  7205 consensus_queue.cc:1048] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.413297  7323 leader_election.cc:290] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:49.414474  7097 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:49.414855  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:49.420745  7097 tablet_bootstrap.cc:654] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.423159  6725 catalog_manager.cc:5697] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:49.424520  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:49.429109  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:49.429759  7048 raft_consensus.cc:3060] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.430097  7096 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:49.433710  7111 raft_consensus.cc:493] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.434243  7111 raft_consensus.cc:515] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.436400  7096 tablet_bootstrap.cc:654] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.436535  7111 leader_election.cc:290] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:49.438593  7356 raft_consensus.cc:493] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.439200  7356 raft_consensus.cc:515] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.441248  7356 leader_election.cc:290] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.441885  7094 tablet_bootstrap.cc:654] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.444649  6793 leader_election.cc:304] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:49.446523  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:49.447077  6901 raft_consensus.cc:3060] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.447420  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4c913d08ea42b09038a2a80ecfa3b6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:49.448017  6898 raft_consensus.cc:2468] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:49.449487  7045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4c913d08ea42b09038a2a80ecfa3b6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.450103  7045 raft_consensus.cc:2468] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:49.451485  6866 leader_election.cc:304] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:49.452152  7222 raft_consensus.cc:493] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.452749  7222 raft_consensus.cc:515] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.454449  7096 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:49.454902  7359 raft_consensus.cc:2804] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.455355  7359 raft_consensus.cc:493] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.455716  7359 raft_consensus.cc:3060] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.454953  7096 ts_tablet_manager.cc:1403] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.000s
I20260709 12:27:49.460386  7222 leader_election.cc:290] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.460932  7045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e381f83716b64b7982f5f0c9c63d348f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.461026  7096 raft_consensus.cc:359] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.461659  7045 raft_consensus.cc:2468] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:49.462327  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.463151  6826 raft_consensus.cc:2393] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 1: Already voted for candidate 38b3b964f82644c7b654e8042953a74e in this term.
I20260709 12:27:49.463335  6866 leader_election.cc:304] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:49.463521  7094 tablet_bootstrap.cc:492] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:49.464099  7094 ts_tablet_manager.cc:1403] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.004s
I20260709 12:27:49.465039  7277 raft_consensus.cc:2804] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.465240  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:49.465500  7277 raft_consensus.cc:493] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.465955  7277 raft_consensus.cc:3060] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.471299  7048 raft_consensus.cc:2468] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:49.472028  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4c913d08ea42b09038a2a80ecfa3b6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.472659  6824 raft_consensus.cc:2468] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:49.474047  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e381f83716b64b7982f5f0c9c63d348f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.474670  6824 raft_consensus.cc:2468] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:49.475766  7094 raft_consensus.cc:359] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.478135  7094 raft_consensus.cc:385] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.478433  7094 raft_consensus.cc:740] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.482039  7094 consensus_queue.cc:260] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.461795  7096 raft_consensus.cc:385] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.483682  7096 raft_consensus.cc:740] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.496524  7094 ts_tablet_manager.cc:1434] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 12:27:49.497661  7094 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:49.498658  7096 consensus_queue.cc:260] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.500260  7359 raft_consensus.cc:515] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.500918  7096 ts_tablet_manager.cc:1434] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 12:27:49.503019  7096 tablet_bootstrap.cc:492] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:49.506032  7013 leader_election.cc:304] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:49.507179  7322 raft_consensus.cc:2749] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.507620  6793 leader_election.cc:304] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:49.492101  7263 raft_consensus.cc:2749] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.510010  7111 raft_consensus.cc:2804] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.510422  7111 raft_consensus.cc:493] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.510812  7111 raft_consensus.cc:3060] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.504418  6901 raft_consensus.cc:2468] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:49.519182  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4c913d08ea42b09038a2a80ecfa3b6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:49.519754  7050 raft_consensus.cc:3060] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.480388  7277 raft_consensus.cc:515] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.536487  7050 raft_consensus.cc:2468] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:49.514321  7359 leader_election.cc:290] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.516929  7322 raft_consensus.cc:493] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.539559  7322 raft_consensus.cc:515] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.540350  7428 raft_consensus.cc:493] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.541397  7315 consensus_queue.cc:1048] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.542403  7356 consensus_queue.cc:1048] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.540894  7111 raft_consensus.cc:515] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.546608  6866 leader_election.cc:304] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:49.548555  7094 tablet_bootstrap.cc:654] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.530143  6791 leader_election.cc:304] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:49.517326  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4c913d08ea42b09038a2a80ecfa3b6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.549796  7428 raft_consensus.cc:515] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.551453  7111 leader_election.cc:290] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:49.558904  7111 raft_consensus.cc:2804] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:49.559360  7111 raft_consensus.cc:697] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:49.560065  7111 consensus_queue.cc:237] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.571403  7277 leader_election.cc:290] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.579178  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4c913d08ea42b09038a2a80ecfa3b6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:49.580116  6824 raft_consensus.cc:2393] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 99db41e57175426b9ceb11625b4455d9 in current term 1: Already voted for candidate 38b3b964f82644c7b654e8042953a74e in this term.
I20260709 12:27:49.581549  7047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e381f83716b64b7982f5f0c9c63d348f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:49.582346  7047 raft_consensus.cc:3060] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.587461  7095 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:49.587984  7095 ts_tablet_manager.cc:1403] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.187s	user 0.017s	sys 0.018s
I20260709 12:27:49.562736  7428 leader_election.cc:290] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:49.554906  7427 raft_consensus.cc:2804] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:49.577384  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4c913d08ea42b09038a2a80ecfa3b6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:49.592021  7048 raft_consensus.cc:2393] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:49.598572  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718060ceb1df4264ac14b8e4c8f9c007" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:49.599328  6976 raft_consensus.cc:2468] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:49.591748  7413 raft_consensus.cc:493] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.605800  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718060ceb1df4264ac14b8e4c8f9c007" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.595613  7427 raft_consensus.cc:697] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:49.597339  7222 raft_consensus.cc:493] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.615746  7096 tablet_bootstrap.cc:654] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.618106  7097 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:49.619534  7097 ts_tablet_manager.cc:1403] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.205s	user 0.024s	sys 0.023s
I20260709 12:27:49.622128  7095 raft_consensus.cc:359] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.621969  7427 consensus_queue.cc:237] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.626236  7094 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:49.611976  7413 raft_consensus.cc:515] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.626798  7094 ts_tablet_manager.cc:1403] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.129s	user 0.029s	sys 0.007s
I20260709 12:27:49.622813  6865 leader_election.cc:304] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:49.626751  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92afb4b5d234fd48fe0f24ec262964a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:49.629416  7413 leader_election.cc:290] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:49.629573  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:49.623636  7222 raft_consensus.cc:515] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.631165  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.631757  7097 raft_consensus.cc:359] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.632387  7222 leader_election.cc:290] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
W20260709 12:27:49.633395  6940 leader_election.cc:343] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.635079  7359 consensus_queue.cc:1048] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:49.624934  7095 raft_consensus.cc:385] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.635816  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.637763  7094 raft_consensus.cc:359] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.638967  7428 raft_consensus.cc:2804] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.639119  7094 raft_consensus.cc:385] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.639560  7094 raft_consensus.cc:740] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
W20260709 12:27:49.639636  7429 log.cc:927] Time spent T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:27:49.636085  7095 raft_consensus.cc:740] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.632418  7097 raft_consensus.cc:385] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.641820  7094 consensus_queue.cc:260] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.642253  7095 consensus_queue.cc:260] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.641957  7097 raft_consensus.cc:740] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.640386  7322 leader_election.cc:290] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.615983  7047 raft_consensus.cc:2468] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:49.640789  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:49.639499  7428 raft_consensus.cc:493] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.647389  7094 ts_tablet_manager.cc:1434] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:27:49.646333  7428 raft_consensus.cc:3060] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.650864  6866 leader_election.cc:304] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:49.652465  7095 ts_tablet_manager.cc:1434] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.002s
I20260709 12:27:49.653496  7094 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:49.651866  7097 consensus_queue.cc:260] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.656373  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92afb4b5d234fd48fe0f24ec262964a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.654486  7277 raft_consensus.cc:2804] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:49.653460  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4c913d08ea42b09038a2a80ecfa3b6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:49.658008  7044 raft_consensus.cc:2393] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
W20260709 12:27:49.658109  7013 leader_election.cc:343] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:49.661170  6942 leader_election.cc:343] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.661631  6942 leader_election.cc:304] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:49.662046  7428 raft_consensus.cc:515] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.663902  7318 raft_consensus.cc:2749] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.664086  7428 leader_election.cc:290] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:49.673103  6865 leader_election.cc:304] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:49.674914  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718060ceb1df4264ac14b8e4c8f9c007" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:49.675525  6974 raft_consensus.cc:3060] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.666818  6976 raft_consensus.cc:2468] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:49.683799  7277 raft_consensus.cc:697] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:49.684759  7277 consensus_queue.cc:237] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:49.687074  6721 catalog_manager.cc:5697] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:49.689057  7427 raft_consensus.cc:2749] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.667254  7095 tablet_bootstrap.cc:492] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:49.700387  6791 leader_election.cc:304] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:49.701128  7013 leader_election.cc:304] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
W20260709 12:27:49.702940  6865 leader_election.cc:343] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.703771  7322 raft_consensus.cc:2804] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.704190  7322 raft_consensus.cc:493] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.704524  7322 raft_consensus.cc:3060] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.704777  7097 ts_tablet_manager.cc:1434] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.000s
I20260709 12:27:49.690559  7222 raft_consensus.cc:493] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.705232  7222 raft_consensus.cc:515] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.714727  6974 raft_consensus.cc:2468] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:49.716531  7097 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:49.716849  7094 tablet_bootstrap.cc:654] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.722668  6865 leader_election.cc:304] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:49.717111  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e381f83716b64b7982f5f0c9c63d348f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:49.723476  6824 raft_consensus.cc:3060] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.723735  7359 raft_consensus.cc:2804] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:49.724284  7359 raft_consensus.cc:697] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:49.723954  6725 catalog_manager.cc:5697] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:49.725229  7359 consensus_queue.cc:237] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.706941  7430 raft_consensus.cc:2749] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.744670  7096 tablet_bootstrap.cc:492] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:49.748242  7096 ts_tablet_manager.cc:1403] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.245s	user 0.007s	sys 0.017s
I20260709 12:27:49.744234  6825 raft_consensus.cc:3060] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.759003  7097 tablet_bootstrap.cc:654] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.763513  7154 raft_consensus.cc:493] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.764169  7154 raft_consensus.cc:515] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.774183  7154 leader_election.cc:290] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:49.750650  7322 raft_consensus.cc:515] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.780812  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718060ceb1df4264ac14b8e4c8f9c007" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:49.791319  7096 raft_consensus.cc:359] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.794348  7096 raft_consensus.cc:385] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:27:49.796098  6865 leader_election.cc:343] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.794608  7096 raft_consensus.cc:740] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.802194  7096 consensus_queue.cc:260] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.803103  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfa9e71770a4e3dac4348d84a9cc2d6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:49.804175  7096 ts_tablet_manager.cc:1434] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.056s	user 0.000s	sys 0.007s
I20260709 12:27:49.810487  7222 leader_election.cc:290] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:49.811632  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:49.813118  6865 leader_election.cc:343] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.813203  6823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:49.815469  6724 catalog_manager.cc:5697] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:49.804841  6942 leader_election.cc:343] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.820742  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92afb4b5d234fd48fe0f24ec262964a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:49.821359  6975 raft_consensus.cc:3060] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.822486  7413 consensus_queue.cc:1048] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.825290  6824 raft_consensus.cc:2468] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:49.830906  7096 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:49.832391  7095 tablet_bootstrap.cc:654] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:49.844554  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:49.852208  7154 consensus_queue.cc:1048] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.846164  7322 leader_election.cc:290] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:49.851567  6898 raft_consensus.cc:3060] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.864079  7097 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:49.864455  7097 ts_tablet_manager.cc:1403] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.148s	user 0.007s	sys 0.014s
I20260709 12:27:49.858162  7047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfa9e71770a4e3dac4348d84a9cc2d6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:49.872371  7047 raft_consensus.cc:2468] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
W20260709 12:27:49.860925  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 45623.5us
I20260709 12:27:49.878800  7094 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:49.882926  7094 ts_tablet_manager.cc:1403] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.230s	user 0.013s	sys 0.018s
W20260709 12:27:49.873564  6865 leader_election.cc:343] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.884991  6865 leader_election.cc:304] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:49.886072  7094 raft_consensus.cc:359] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.886778  7094 raft_consensus.cc:385] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.887032  7094 raft_consensus.cc:740] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.887459  7359 raft_consensus.cc:2749] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:49.880072  6975 raft_consensus.cc:2468] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:49.896327  7097 raft_consensus.cc:359] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.897020  7094 consensus_queue.cc:260] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.900271  7096 tablet_bootstrap.cc:654] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:49.897012  7097 raft_consensus.cc:385] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:49.901181  7097 raft_consensus.cc:740] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:49.913178  7013 leader_election.cc:304] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:49.915781  7322 raft_consensus.cc:2804] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:49.916177  7413 consensus_queue.cc:1048] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.916344  6821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c92afb4b5d234fd48fe0f24ec262964a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:49.913897  6940 leader_election.cc:304] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:49.919595  7315 raft_consensus.cc:2804] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:49.920022  7315 raft_consensus.cc:493] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.920338  7315 raft_consensus.cc:3060] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:49.916345  7322 raft_consensus.cc:697] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:49.922580  7322 consensus_queue.cc:237] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:49.911428  7097 consensus_queue.cc:260] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:49.934046  7094 ts_tablet_manager.cc:1434] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 12:27:49.935384  7094 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:49.945058  7097 ts_tablet_manager.cc:1434] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
I20260709 12:27:49.926357  7263 raft_consensus.cc:493] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.955834  7097 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:49.962088  7111 raft_consensus.cc:493] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.942867  7323 raft_consensus.cc:493] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.979363  7323 raft_consensus.cc:515] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:49.981570  7323 leader_election.cc:290] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
W20260709 12:27:49.982911  7013 leader_election.cc:343] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.983304  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:49.984987  7016 leader_election.cc:343] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:49.985397  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:49.986130  7110 consensus_queue.cc:1048] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:49.986048  6973 raft_consensus.cc:2468] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:49.962790  7111 raft_consensus.cc:515] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.001204  6721 catalog_manager.cc:5697] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:50.010448  7443 raft_consensus.cc:493] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.011204  7443 raft_consensus.cc:515] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.013803  7443 leader_election.cc:290] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:49.958725  7263 raft_consensus.cc:515] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.024299  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:50.025833  7205 consensus_queue.cc:1048] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:49.975252  7315 raft_consensus.cc:515] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.033530  7111 leader_election.cc:290] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:50.029999  7315 leader_election.cc:290] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:50.043823  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:50.044793  7050 raft_consensus.cc:2468] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
W20260709 12:27:50.048223  7016 leader_election.cc:343] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.048904  6791 leader_election.cc:304] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:50.049890  7111 raft_consensus.cc:2804] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:50.050232  7111 raft_consensus.cc:493] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.050556  7111 raft_consensus.cc:3060] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.050776  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfa9e71770a4e3dac4348d84a9cc2d6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:50.077127  7111 raft_consensus.cc:515] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.079074  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:50.080951  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:50.081383  7111 leader_election.cc:290] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
W20260709 12:27:50.081543  6942 leader_election.cc:343] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.082599  7013 leader_election.cc:304] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:50.083943  7095 tablet_bootstrap.cc:492] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:50.084450  7095 ts_tablet_manager.cc:1403] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.417s	user 0.039s	sys 0.011s
I20260709 12:27:50.081641  7048 raft_consensus.cc:3060] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.087795  7443 raft_consensus.cc:2804] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:50.098567  7096 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:50.102097  7096 ts_tablet_manager.cc:1403] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.297s	user 0.014s	sys 0.051s
I20260709 12:27:50.102770  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfa9e71770a4e3dac4348d84a9cc2d6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:50.103437  7050 raft_consensus.cc:3060] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.101656  7443 raft_consensus.cc:493] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.110582  7443 raft_consensus.cc:3060] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.120986  7263 leader_election.cc:290] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:50.132927  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:50.133200  7095 raft_consensus.cc:359] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.133912  7095 raft_consensus.cc:385] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.125061  7096 raft_consensus.cc:359] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.134079  6826 raft_consensus.cc:2393] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 1: Already voted for candidate 38b3b964f82644c7b654e8042953a74e in this term.
I20260709 12:27:50.134178  7095 raft_consensus.cc:740] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.134485  7096 raft_consensus.cc:385] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.141129  7097 tablet_bootstrap.cc:654] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.142828  7096 raft_consensus.cc:740] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.144487  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718060ceb1df4264ac14b8e4c8f9c007" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:50.145330  6973 raft_consensus.cc:2393] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:50.146598  7096 consensus_queue.cc:260] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.151882  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:50.159912  7277 consensus_queue.cc:1048] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:50.163687  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718060ceb1df4264ac14b8e4c8f9c007" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:50.164264  7446 log.cc:927] Time spent T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 12:27:50.135954  6793 leader_election.cc:343] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.141222  7443 raft_consensus.cc:515] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.142418  7095 consensus_queue.cc:260] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.181366  7094 tablet_bootstrap.cc:654] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.179989  7096 ts_tablet_manager.cc:1434] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 12:27:50.182693  7095 ts_tablet_manager.cc:1434] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.098s	user 0.007s	sys 0.002s
I20260709 12:27:50.186491  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
W20260709 12:27:50.187919  7016 leader_election.cc:343] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.185017  7050 raft_consensus.cc:2468] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:50.189581  7443 leader_election.cc:290] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:50.188362  7096 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:50.193356  7095 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:50.195021  7263 consensus_queue.cc:1048] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:27:50.191196  7277 consensus_queue.cc:1048] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:27:50.219008  6725 catalog_manager.cc:5697] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:50.218456  6793 leader_election.cc:343] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.226805  7427 consensus_queue.cc:1048] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:27:50.243594  6823 raft_consensus.cc:3060] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.235023  7013 leader_election.cc:304] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:50.254532  7097 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:50.255089  7097 ts_tablet_manager.cc:1403] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.300s	user 0.026s	sys 0.013s
I20260709 12:27:50.239846  7048 raft_consensus.cc:2468] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:50.258461  7435 raft_consensus.cc:2749] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:50.241356  7222 consensus_queue.cc:1048] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:27:50.259976  6791 leader_election.cc:304] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:50.244525  7154 raft_consensus.cc:493] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.261242  7154 raft_consensus.cc:515] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.264650  7154 leader_election.cc:290] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:50.257836  7097 raft_consensus.cc:359] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.267555  7097 raft_consensus.cc:385] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.267948  7097 raft_consensus.cc:740] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.269205  7097 consensus_queue.cc:260] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.231776  6940 leader_election.cc:304] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:50.275318  7111 raft_consensus.cc:2804] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:50.277390  7111 raft_consensus.cc:697] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:50.278177  7111 consensus_queue.cc:237] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.284917  7154 raft_consensus.cc:2804] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:50.285424  7154 raft_consensus.cc:697] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:50.288340  7097 ts_tablet_manager.cc:1434] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
W20260709 12:27:50.283383  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 52262.1us
I20260709 12:27:50.297501  7096 tablet_bootstrap.cc:654] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.305030  7095 tablet_bootstrap.cc:654] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.291594  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fa5eb82f684b449d82176587520e4d" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:50.310074  6824 raft_consensus.cc:2468] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
I20260709 12:27:50.286377  7154 consensus_queue.cc:237] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.244796  6793 leader_election.cc:304] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:50.277207  7222 consensus_queue.cc:1048] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 12:27:50.322918  7097 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:50.323983  7111 raft_consensus.cc:2749] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:50.326928  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:50.327551  6974 raft_consensus.cc:3060] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:50.313050  6940 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 86318.5us
I20260709 12:27:50.355881  7045 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fa5eb82f684b449d82176587520e4d" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:50.356542  7045 raft_consensus.cc:2468] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 0.
W20260709 12:27:50.357401  7457 log.cc:927] Time spent T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:27:50.345999  7094 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:50.348999  6939 leader_election.cc:304] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:50.343237  7277 consensus_queue.cc:1048] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
W20260709 12:27:50.343954  6793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 105304us
I20260709 12:27:50.369423  7094 ts_tablet_manager.cc:1403] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.434s	user 0.012s	sys 0.005s
I20260709 12:27:50.376788  7315 raft_consensus.cc:2804] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:50.387933  7315 raft_consensus.cc:493] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.398756  7315 raft_consensus.cc:3060] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.409727  6974 raft_consensus.cc:2468] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
W20260709 12:27:50.423454  6940 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 63208us
I20260709 12:27:50.439666  7315 raft_consensus.cc:515] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.465574  7435 raft_consensus.cc:493] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.466161  7435 raft_consensus.cc:515] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.468355  7435 leader_election.cc:290] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:50.467253  7013 leader_election.cc:304] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:50.470566  7443 raft_consensus.cc:2804] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:50.471060  7443 raft_consensus.cc:697] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:50.470798  7094 raft_consensus.cc:359] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.472012  7094 raft_consensus.cc:385] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.472365  7094 raft_consensus.cc:740] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.454487  6724 catalog_manager.cc:5697] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:50.473084  7094 consensus_queue.cc:260] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.477886  7443 consensus_queue.cc:237] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.481280  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:50.483909  7016 leader_election.cc:343] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.486956  7097 tablet_bootstrap.cc:654] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.493861  7094 ts_tablet_manager.cc:1434] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.120s	user 0.007s	sys 0.000s
W20260709 12:27:50.494885  6940 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 43724.4us
I20260709 12:27:50.498708  7095 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:50.499212  7095 ts_tablet_manager.cc:1403] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.310s	user 0.016s	sys 0.010s
I20260709 12:27:50.500052  7096 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:50.500558  7096 ts_tablet_manager.cc:1403] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.313s	user 0.025s	sys 0.011s
I20260709 12:27:50.500604  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fa5eb82f684b449d82176587520e4d" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:50.501351  6826 raft_consensus.cc:3060] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.499223  7315 leader_election.cc:290] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:50.508795  7095 raft_consensus.cc:359] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.511759  7095 raft_consensus.cc:385] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.512032  7095 raft_consensus.cc:740] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.512666  7095 consensus_queue.cc:260] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.514954  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fa5eb82f684b449d82176587520e4d" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:50.516280  7048 raft_consensus.cc:3060] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.525079  7322 consensus_queue.cc:1048] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:50.525434  7095 ts_tablet_manager.cc:1434] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:27:50.526520  7095 tablet_bootstrap.cc:492] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:50.526901  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:50.525334  6721 catalog_manager.cc:5697] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:50.530402  7464 raft_consensus.cc:493] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.530856  7464 raft_consensus.cc:515] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.517035  6826 raft_consensus.cc:2468] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:50.532398  6939 leader_election.cc:304] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:50.532915  7464 leader_election.cc:290] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:50.533579  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfa9e71770a4e3dac4348d84a9cc2d6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:50.534036  7095 tablet_bootstrap.cc:654] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.534482  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dfa9e71770a4e3dac4348d84a9cc2d6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:50.534368  7154 raft_consensus.cc:2804] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:50.536059  7048 raft_consensus.cc:2468] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:50.535326  7044 raft_consensus.cc:2393] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99db41e57175426b9ceb11625b4455d9 in current term 1: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:50.537453  7154 raft_consensus.cc:697] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:50.540452  7096 raft_consensus.cc:359] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.541280  7096 raft_consensus.cc:385] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.541594  7096 raft_consensus.cc:740] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.538204  7154 consensus_queue.cc:237] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.542948  7094 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:50.542343  7096 consensus_queue.cc:260] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.548748  6866 leader_election.cc:304] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:50.551131  7222 raft_consensus.cc:493] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.551191  7427 raft_consensus.cc:3060] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.552135  7222 raft_consensus.cc:515] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:27:50.554203  7013 leader_election.cc:343] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.554951  7222 leader_election.cc:290] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:50.556826  6823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:50.559676  7094 tablet_bootstrap.cc:654] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.559886  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:50.560668  7044 raft_consensus.cc:2393] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99db41e57175426b9ceb11625b4455d9 in current term 1: Already voted for candidate 38b3b964f82644c7b654e8042953a74e in this term.
I20260709 12:27:50.562690  6866 leader_election.cc:304] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:50.554736  7013 leader_election.cc:304] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:50.565454  7096 ts_tablet_manager.cc:1434] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 12:27:50.566195  7443 raft_consensus.cc:2749] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:50.566524  7096 tablet_bootstrap.cc:492] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:50.569352  7222 raft_consensus.cc:3060] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.568686  6722 catalog_manager.cc:5697] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:50.573873  7427 raft_consensus.cc:2749] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:50.579277  7013 consensus_peers.cc:597] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:50.583850  7097 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:50.584270  7097 ts_tablet_manager.cc:1403] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.269s	user 0.012s	sys 0.003s
I20260709 12:27:50.587322  7097 raft_consensus.cc:359] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.588183  7097 raft_consensus.cc:385] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.588459  7097 raft_consensus.cc:740] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.589114  7097 consensus_queue.cc:260] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.599808  7097 ts_tablet_manager.cc:1434] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.006s
I20260709 12:27:50.622197  7222 raft_consensus.cc:2749] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:50.624228  7097 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:50.634160  7096 tablet_bootstrap.cc:654] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.626715  7094 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:50.636361  7094 ts_tablet_manager.cc:1403] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.099s	user 0.011s	sys 0.006s
I20260709 12:27:50.623909  6721 catalog_manager.cc:5697] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 0 to 1, leader changed from <none> to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 1 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:50.643566  7095 tablet_bootstrap.cc:492] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:50.639957  7094 raft_consensus.cc:359] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.644380  7094 raft_consensus.cc:385] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.644848  7094 raft_consensus.cc:740] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.645486  7094 consensus_queue.cc:260] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.647764  7094 ts_tablet_manager.cc:1434] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:27:50.648725  7094 tablet_bootstrap.cc:492] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:50.644235  7095 ts_tablet_manager.cc:1403] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.118s	user 0.026s	sys 0.003s
I20260709 12:27:50.654353  7094 tablet_bootstrap.cc:654] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.654358  7095 raft_consensus.cc:359] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.655413  7095 raft_consensus.cc:385] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.659018  7095 raft_consensus.cc:740] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.660923  7096 tablet_bootstrap.cc:492] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:50.663435  7096 ts_tablet_manager.cc:1403] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.097s	user 0.018s	sys 0.014s
I20260709 12:27:50.663722  7094 tablet_bootstrap.cc:492] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:50.664139  7094 ts_tablet_manager.cc:1403] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.004s
I20260709 12:27:50.665890  7096 raft_consensus.cc:359] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.666566  7096 raft_consensus.cc:385] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.666811  7096 raft_consensus.cc:740] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.667006  7094 raft_consensus.cc:359] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.667443  7096 consensus_queue.cc:260] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.668011  7094 raft_consensus.cc:385] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.662518  7095 consensus_queue.cc:260] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.672508  7094 raft_consensus.cc:740] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.662091  7097 tablet_bootstrap.cc:654] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.673259  7094 consensus_queue.cc:260] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.673976  7095 ts_tablet_manager.cc:1434] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 12:27:50.674921  7095 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:50.669863  7096 ts_tablet_manager.cc:1434] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:27:50.681109  7096 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:50.681593  7094 ts_tablet_manager.cc:1434] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:27:50.682178  7095 tablet_bootstrap.cc:654] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.682682  7094 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:50.691692  7096 tablet_bootstrap.cc:654] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.696528  7321 raft_consensus.cc:493] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.697208  7321 raft_consensus.cc:515] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.701682  7321 leader_election.cc:290] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:50.701958  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:50.703090  6976 raft_consensus.cc:2468] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:50.704411  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:50.705049  6898 raft_consensus.cc:2468] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:50.705658  6790 leader_election.cc:304] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:50.707114  7321 raft_consensus.cc:2804] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:50.707489  7321 raft_consensus.cc:493] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.707821  7321 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.708501  7094 tablet_bootstrap.cc:654] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.721596  7111 consensus_queue.cc:1048] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:50.727888  7321 raft_consensus.cc:515] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.732474  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:50.733193  6898 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:50.840943  7485 log.cc:927] Time spent T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9: Append to log took a long time: real 0.096s	user 0.002s	sys 0.000s
I20260709 12:27:50.852874  7096 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:50.853489  7096 ts_tablet_manager.cc:1403] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.025s
I20260709 12:27:50.855461  7095 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:50.856025  7095 ts_tablet_manager.cc:1403] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.181s	user 0.051s	sys 0.046s
I20260709 12:27:50.856165  7096 raft_consensus.cc:359] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.856897  7096 raft_consensus.cc:385] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.857153  7096 raft_consensus.cc:740] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.859515  7097 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:50.860030  7097 ts_tablet_manager.cc:1403] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.236s	user 0.029s	sys 0.022s
I20260709 12:27:50.859350  7095 raft_consensus.cc:359] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.862167  7095 raft_consensus.cc:385] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.862537  7095 raft_consensus.cc:740] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.863564  7094 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:50.863246  7095 consensus_queue.cc:260] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.864032  7094 ts_tablet_manager.cc:1403] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.182s	user 0.027s	sys 0.007s
I20260709 12:27:50.865257  7095 ts_tablet_manager.cc:1434] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:27:50.866349  7095 tablet_bootstrap.cc:492] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:50.866648  7097 raft_consensus.cc:359] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.867348  7097 raft_consensus.cc:385] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.867619  7097 raft_consensus.cc:740] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.868247  7097 consensus_queue.cc:260] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.869374  7094 raft_consensus.cc:359] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.870071  7094 raft_consensus.cc:385] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.870347  7094 raft_consensus.cc:740] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.870939  7094 consensus_queue.cc:260] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.872324  7094 ts_tablet_manager.cc:1434] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:27:50.873196  7094 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:50.873246  7095 tablet_bootstrap.cc:654] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.879289  7097 ts_tablet_manager.cc:1434] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:27:50.880270  7097 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:50.857757  7096 consensus_queue.cc:260] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.884954  6898 raft_consensus.cc:2468] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:50.890939  7321 leader_election.cc:290] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:50.892493  7096 ts_tablet_manager.cc:1434] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.000s
I20260709 12:27:50.920197  7096 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76: Bootstrap starting.
I20260709 12:27:50.896576  7322 raft_consensus.cc:493] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.903842  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:50.904331  7435 raft_consensus.cc:493] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.922712  6974 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.922842  7435 raft_consensus.cc:515] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:50.924322  7315 raft_consensus.cc:493] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.924825  7315 raft_consensus.cc:515] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.912526  7094 tablet_bootstrap.cc:654] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.925792  7096 tablet_bootstrap.cc:654] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.910153  7097 tablet_bootstrap.cc:654] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.926963  7315 leader_election.cc:290] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:50.925115  7435 leader_election.cc:290] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:50.906164  6899 raft_consensus.cc:3060] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.932596  7097 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:50.933058  7097 ts_tablet_manager.cc:1403] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 12:27:50.933869  6974 raft_consensus.cc:2468] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:50.934569  7435 raft_consensus.cc:493] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.935127  7435 raft_consensus.cc:515] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.935668  6826 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b065e90800f40379b501a6238c6fdcd" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:50.935812  7097 raft_consensus.cc:359] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.922610  7322 raft_consensus.cc:515] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.936514  7097 raft_consensus.cc:385] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.937482  7097 raft_consensus.cc:740] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.938261  7322 leader_election.cc:290] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:50.938113  7097 consensus_queue.cc:260] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.939116  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ecb64e39814c7a961ebb058d596cbc" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:50.939839  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df4f6356b53461e89321a089ab9b65f" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:50.939960  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df4f6356b53461e89321a089ab9b65f" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:50.941279  7016 leader_election.cc:343] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.941431  7097 ts_tablet_manager.cc:1434] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:27:50.942682  7097 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:50.942526  6976 raft_consensus.cc:2468] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:50.943444  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df4f6356b53461e89321a089ab9b65f" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:50.944075  6973 raft_consensus.cc:2468] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:50.944303  6793 leader_election.cc:304] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:50.952860  7205 raft_consensus.cc:2804] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:50.954452  7096 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76: No bootstrap required, opened a new log
I20260709 12:27:50.954931  7096 ts_tablet_manager.cc:1403] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.007s
I20260709 12:27:50.953334  7205 raft_consensus.cc:697] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:50.956000  7205 consensus_queue.cc:237] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.957139  7095 tablet_bootstrap.cc:492] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:50.957633  7095 ts_tablet_manager.cc:1403] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.091s	user 0.015s	sys 0.011s
I20260709 12:27:50.957834  7096 raft_consensus.cc:359] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.960246  7094 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:50.960731  7094 ts_tablet_manager.cc:1403] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.088s	user 0.009s	sys 0.000s
I20260709 12:27:50.961917  6823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ecb64e39814c7a961ebb058d596cbc" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:50.963510  7094 raft_consensus.cc:359] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.964226  7094 raft_consensus.cc:385] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.964519  7094 raft_consensus.cc:740] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.965188  7094 consensus_queue.cc:260] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:50.945303  7435 leader_election.cc:290] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:50.946736  7013 leader_election.cc:304] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
W20260709 12:27:50.968298  6942 leader_election.cc:343] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.970489  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b065e90800f40379b501a6238c6fdcd" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:50.972844  7094 ts_tablet_manager.cc:1434] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:27:50.973845  7094 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21: Bootstrap starting.
I20260709 12:27:50.960392  7095 raft_consensus.cc:359] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.976864  7095 raft_consensus.cc:385] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.977152  7095 raft_consensus.cc:740] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.979030  7097 tablet_bootstrap.cc:654] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.979849  7263 consensus_queue.cc:1048] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:50.981554  7096 raft_consensus.cc:385] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:50.981911  7096 raft_consensus.cc:740] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Initialized, Role: FOLLOWER
I20260709 12:27:50.982559  7096 consensus_queue.cc:260] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:50.984292  7466 raft_consensus.cc:493] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.984607  7094 tablet_bootstrap.cc:654] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:50.984822  7466 raft_consensus.cc:515] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:50.987035  7466 leader_election.cc:290] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
W20260709 12:27:50.987668  7016 leader_election.cc:343] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.988070  7315 consensus_queue.cc:1048] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:50.991110  7435 consensus_queue.cc:1048] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:50.991662  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
W20260709 12:27:50.992477  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 45832.9us
I20260709 12:27:50.993428  7013 leader_election.cc:304] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:50.994136  7322 raft_consensus.cc:2804] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:50.994387  7443 raft_consensus.cc:2804] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:50.994578  7322 raft_consensus.cc:493] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.994854  7443 raft_consensus.cc:493] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.995064  7322 raft_consensus.cc:3060] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:50.995236  7443 raft_consensus.cc:3060] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:50.998699  7013 leader_election.cc:343] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:50.999205  7013 leader_election.cc:304] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:51.000162  7451 raft_consensus.cc:2749] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:51.001719  7096 ts_tablet_manager.cc:1434] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76: Time spent starting tablet: real 0.046s	user 0.003s	sys 0.003s
W20260709 12:27:51.003141  7013 leader_election.cc:343] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:51.006984  7322 raft_consensus.cc:515] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:51.009032  7322 leader_election.cc:290] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:51.011149  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ecb64e39814c7a961ebb058d596cbc" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:51.011713  6976 raft_consensus.cc:3060] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.013955  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df4f6356b53461e89321a089ab9b65f" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:51.017287  7097 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:51.017766  7097 ts_tablet_manager.cc:1403] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.011s
I20260709 12:27:51.020535  7097 raft_consensus.cc:359] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:51.021173  7097 raft_consensus.cc:385] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:51.020702  7443 raft_consensus.cc:515] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:51.021483  7097 raft_consensus.cc:740] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:51.023391  7048 raft_consensus.cc:2393] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 1: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:50.992449  6974 raft_consensus.cc:2468] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:51.027858  6865 leader_election.cc:304] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:51.028209  7097 consensus_queue.cc:260] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
W20260709 12:27:51.029356  6791 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 53216.6us
I20260709 12:27:51.148384  7451 consensus_queue.cc:1048] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:51.152978  6976 raft_consensus.cc:2468] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:51.029153  7466 raft_consensus.cc:2804] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:51.156672  7466 raft_consensus.cc:493] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fb7d0187b544dfdb8487ecb2ff24a76)
I20260709 12:27:51.157104  7466 raft_consensus.cc:3060] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.030063  7315 raft_consensus.cc:493] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.030673  7095 consensus_queue.cc:260] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:51.031123  7154 raft_consensus.cc:493] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.084836  7430 raft_consensus.cc:493] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.136087  7443 leader_election.cc:290] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:51.138171  7094 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21: No bootstrap required, opened a new log
I20260709 12:27:51.163295  7094 ts_tablet_manager.cc:1403] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21: Time spent bootstrapping tablet: real 0.190s	user 0.018s	sys 0.012s
I20260709 12:27:51.145540  6823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ecb64e39814c7a961ebb058d596cbc" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
W20260709 12:27:51.146489  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 111034us
I20260709 12:27:51.159929  7154 raft_consensus.cc:515] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:51.167753  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:51.159204  7315 raft_consensus.cc:515] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:51.164757  7095 ts_tablet_manager.cc:1434] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.207s	user 0.001s	sys 0.004s
I20260709 12:27:51.165444  7430 raft_consensus.cc:515] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:51.161808  7013 leader_election.cc:304] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:51.186637  7094 raft_consensus.cc:359] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:51.187295  7094 raft_consensus.cc:385] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:51.187562  7094 raft_consensus.cc:740] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Initialized, Role: FOLLOWER
I20260709 12:27:51.188186  7094 consensus_queue.cc:260] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:51.190766  6825 raft_consensus.cc:3060] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.195765  7097 ts_tablet_manager.cc:1434] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.178s	user 0.019s	sys 0.033s
I20260709 12:27:51.195590  6721 catalog_manager.cc:5697] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:51.198726  7097 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:51.199489  7430 leader_election.cc:290] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:51.201282  7315 leader_election.cc:290] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:51.179911  6823 raft_consensus.cc:3060] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.203933  7323 raft_consensus.cc:493] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.204592  7323 raft_consensus.cc:515] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:51.203868  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df4f6356b53461e89321a089ab9b65f" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:51.207166  7154 leader_election.cc:290] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:51.207891  7095 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:51.210408  6972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:51.211032  6972 raft_consensus.cc:2468] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:51.211328  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2b62ad0158424696dcdf73faa4f675" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:51.212087  6790 leader_election.cc:304] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:51.213637  7430 raft_consensus.cc:2804] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:51.213995  7430 raft_consensus.cc:493] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.214357  7430 raft_consensus.cc:3060] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.215911  6976 raft_consensus.cc:3060] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.217065  7427 raft_consensus.cc:493] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.217717  7427 raft_consensus.cc:515] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:51.220777  7427 leader_election.cc:290] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:51.223076  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:51.223892  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5df4f6356b53461e89321a089ab9b65f" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:51.224471  6899 raft_consensus.cc:3060] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.224964  7443 raft_consensus.cc:2804] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:51.225548  7443 raft_consensus.cc:697] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:51.226504  7443 consensus_queue.cc:237] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:51.230659  7094 ts_tablet_manager.cc:1434] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21: Time spent starting tablet: real 0.067s	user 0.003s	sys 0.003s
W20260709 12:27:51.224463  7013 leader_election.cc:343] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:51.234908  6899 raft_consensus.cc:2468] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:51.236243  7016 leader_election.cc:304] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:51.237107  7435 raft_consensus.cc:2804] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:51.237583  7435 raft_consensus.cc:697] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:51.238461  7435 consensus_queue.cc:237] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:51.207751  7323 leader_election.cc:290] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
W20260709 12:27:51.250244  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:51.245011  6823 raft_consensus.cc:2468] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:51.252557  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2b62ad0158424696dcdf73faa4f675" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:51.275588  6974 raft_consensus.cc:2468] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 0.
I20260709 12:27:51.277276  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:51.290357  7472 consensus_queue.cc:1048] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:51.290557  6940 leader_election.cc:304] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:51.292732  7472 consensus_queue.cc:1048] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:51.296330  7013 leader_election.cc:304] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:51.296574  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
W20260709 12:27:51.316120  6942 leader_election.cc:343] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:51.278291  7430 raft_consensus.cc:515] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:51.309357  7323 raft_consensus.cc:2804] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:51.340209  7323 raft_consensus.cc:493] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.348789  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:51.338258  6900 raft_consensus.cc:2468] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
W20260709 12:27:51.395130  6793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 96968.2us
I20260709 12:27:51.391337  7323 raft_consensus.cc:3060] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.430980  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:51.431872  6973 raft_consensus.cc:2468] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:51.421622  6824 raft_consensus.cc:2393] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99db41e57175426b9ceb11625b4455d9 in current term 1: Already voted for candidate 38b3b964f82644c7b654e8042953a74e in this term.
I20260709 12:27:51.344386  7466 raft_consensus.cc:515] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:51.504273  7095 tablet_bootstrap.cc:654] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:51.556964  7097 tablet_bootstrap.cc:654] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:51.566865  7505 log.cc:927] Time spent T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:27:51.567719  7466 leader_election.cc:290] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
W20260709 12:27:51.668725  7503 log.cc:927] Time spent T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:27:51.686682  7323 raft_consensus.cc:515] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:51.689347  7323 leader_election.cc:290] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:51.706015  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:51.686764  6976 raft_consensus.cc:2468] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
W20260709 12:27:51.710300  7511 log.cc:927] Time spent T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:27:51.709957  7452 raft_consensus.cc:2749] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:51.721099  6791 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 204595us
I20260709 12:27:51.743230  6865 leader_election.cc:304] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:51.744129  7507 raft_consensus.cc:2804] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:51.744509  7507 raft_consensus.cc:493] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.744827  7507 raft_consensus.cc:3060] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.755553  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:51.763666  6827 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:51.808717  6818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2b62ad0158424696dcdf73faa4f675" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
W20260709 12:27:51.812924  7013 leader_election.cc:343] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:51.855875  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 80419.2us
W20260709 12:27:51.860669  6866 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 87067.5us
I20260709 12:27:51.816051  7507 raft_consensus.cc:515] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:51.828349  7522 consensus_queue.cc:1048] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 12:27:51.866313  7095 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:51.867825  7095 ts_tablet_manager.cc:1403] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.660s	user 0.018s	sys 0.036s
I20260709 12:27:51.872552  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:51.876253  7048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:51.880244  7097 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:51.880914  7097 ts_tablet_manager.cc:1403] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.682s	user 0.013s	sys 0.010s
I20260709 12:27:51.884474  7321 consensus_queue.cc:1048] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:51.891714  6902 raft_consensus.cc:2393] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:51.892433  7507 leader_election.cc:290] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:51.894273  6939 leader_election.cc:304] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:51.895174  7508 raft_consensus.cc:2749] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:51.895586  6939 leader_election.cc:343] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:51.896086  6939 leader_election.cc:304] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:51.897243  7495 raft_consensus.cc:2749] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:51.903091  6866 leader_election.cc:304] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:51.906260  6901 raft_consensus.cc:1240] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Rejecting Update request from peer 3fb7d0187b544dfdb8487ecb2ff24a76 for earlier term 1. Current term is 2. Ops: []
I20260709 12:27:51.907792  7536 raft_consensus.cc:2749] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:27:51.923455  7097 raft_consensus.cc:359] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:51.923743  7520 consensus_queue.cc:1048] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:51.924250  7097 raft_consensus.cc:385] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:51.924578  7097 raft_consensus.cc:740] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:51.925225  7097 consensus_queue.cc:260] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:51.940845  7097 ts_tablet_manager.cc:1434] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
W20260709 12:27:51.951009  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 85099.2us
I20260709 12:27:51.954090  7097 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:51.967581  7095 raft_consensus.cc:359] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:51.968577  7095 raft_consensus.cc:385] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:51.968912  7095 raft_consensus.cc:740] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:51.969779  7095 consensus_queue.cc:260] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:51.971369  6972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:51.972254  6972 raft_consensus.cc:3060] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:51.972863  7095 ts_tablet_manager.cc:1434] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.000s
I20260709 12:27:51.978832  7430 leader_election.cc:290] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:51.979794  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:51.981386  6790 leader_election.cc:304] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:51.983450  7321 raft_consensus.cc:2749] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:52.029578  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:52.030453  6824 raft_consensus.cc:2393] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 99db41e57175426b9ceb11625b4455d9 in current term 1: Already voted for candidate 38b3b964f82644c7b654e8042953a74e in this term.
I20260709 12:27:52.043212  6972 raft_consensus.cc:2468] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:52.044442  7095 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
W20260709 12:27:52.053732  7543 log.cc:927] Time spent T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:27:52.054014  6725 catalog_manager.cc:5697] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:52.056418  6725 catalog_manager.cc:5697] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:52.059820  7542 log.cc:927] Time spent T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:27:52.062669  7323 consensus_queue.cc:1059] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:52.063984  7323 raft_consensus.cc:3055] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:52.064313  7323 raft_consensus.cc:740] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:52.065121  7323 consensus_queue.cc:260] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:52.070919  7520 raft_consensus.cc:493] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.071489  7520 raft_consensus.cc:515] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.074182  7520 leader_election.cc:290] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
W20260709 12:27:52.074713  6942 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 165371us
I20260709 12:27:52.075222  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:52.075894  7323 raft_consensus.cc:3060] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.076277  6897 raft_consensus.cc:2468] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:52.081069  6793 leader_election.cc:304] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:52.082020  7530 raft_consensus.cc:2804] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:52.082463  7530 raft_consensus.cc:493] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.082858  7530 raft_consensus.cc:3060] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.083861  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2b62ad0158424696dcdf73faa4f675" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:52.084474  6974 raft_consensus.cc:3060] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.091444  6974 raft_consensus.cc:2468] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:52.092566  7522 consensus_queue.cc:1048] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:52.013425  6939 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 49045.3us
I20260709 12:27:52.109902  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:52.110775  7043 raft_consensus.cc:2468] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:52.113612  7095 tablet_bootstrap.cc:654] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:52.119890  7528 consensus_queue.cc:1048] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:52.127043  7013 leader_election.cc:304] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:52.127786  7522 raft_consensus.cc:2804] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:52.131524  7522 raft_consensus.cc:697] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:52.132696  7522 consensus_queue.cc:237] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:52.133088  7435 consensus_queue.cc:1048] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:52.141677  7531 consensus_queue.cc:1048] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:52.171607  7097 tablet_bootstrap.cc:654] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:52.176754  7095 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:52.177296  7095 ts_tablet_manager.cc:1403] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.133s	user 0.012s	sys 0.001s
I20260709 12:27:52.180068  7095 raft_consensus.cc:359] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:52.180996  7095 raft_consensus.cc:385] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:52.185089  6865 leader_election.cc:304] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:52.186197  7540 raft_consensus.cc:2804] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:52.186702  7540 raft_consensus.cc:697] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:52.187528  7540 consensus_queue.cc:237] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:52.181334  7095 raft_consensus.cc:740] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:52.198925  7095 consensus_queue.cc:260] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:52.222443  7095 ts_tablet_manager.cc:1434] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:27:52.226357  7095 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9: Bootstrap starting.
I20260709 12:27:52.243497  6724 catalog_manager.cc:5697] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 1 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.179592  7530 raft_consensus.cc:515] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.263027  7097 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:52.263554  7097 ts_tablet_manager.cc:1403] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.319s	user 0.012s	sys 0.000s
I20260709 12:27:52.266695  7097 raft_consensus.cc:359] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:52.267354  7097 raft_consensus.cc:385] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:52.267632  7097 raft_consensus.cc:740] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:52.268349  7097 consensus_queue.cc:260] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:52.277925  7530 leader_election.cc:290] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
W20260709 12:27:52.280467  6940 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 45436.3us
I20260709 12:27:52.280922  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:52.281553  6899 raft_consensus.cc:3060] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.280597  7095 tablet_bootstrap.cc:654] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:52.285696  7047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:52.287386  7047 raft_consensus.cc:3060] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.294610  6899 raft_consensus.cc:2468] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:52.303020  6793 leader_election.cc:304] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:52.303915  7529 raft_consensus.cc:2804] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:52.304405  7529 raft_consensus.cc:697] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:52.305166  7529 consensus_queue.cc:237] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.309394  7047 raft_consensus.cc:2468] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:52.311601  7097 ts_tablet_manager.cc:1434] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 12:27:52.316272  7097 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:52.328296  7097 tablet_bootstrap.cc:654] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:52.346859  7095 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9: No bootstrap required, opened a new log
I20260709 12:27:52.347378  7095 ts_tablet_manager.cc:1403] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9: Time spent bootstrapping tablet: real 0.121s	user 0.010s	sys 0.000s
I20260709 12:27:52.350126  7095 raft_consensus.cc:359] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.350807  7095 raft_consensus.cc:385] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:52.351078  7095 raft_consensus.cc:740] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Initialized, Role: FOLLOWER
I20260709 12:27:52.351804  7095 consensus_queue.cc:260] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.352633  7097 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:52.353158  7097 ts_tablet_manager.cc:1403] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.003s
I20260709 12:27:52.356031  7095 ts_tablet_manager.cc:1434] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:27:52.355808  7097 raft_consensus.cc:359] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:52.356586  7097 raft_consensus.cc:385] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:52.356945  7097 raft_consensus.cc:740] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:52.357744  7097 consensus_queue.cc:260] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:52.360121  7097 ts_tablet_manager.cc:1434] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:27:52.361142  7097 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:52.364539  6724 catalog_manager.cc:5697] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.367093  7097 tablet_bootstrap.cc:654] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:52.371754  7097 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: No bootstrap required, opened a new log
I20260709 12:27:52.372169  7097 ts_tablet_manager.cc:1403] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.001s
I20260709 12:27:52.375051  7097 raft_consensus.cc:359] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.375800  7097 raft_consensus.cc:385] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:52.376134  7097 raft_consensus.cc:740] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:52.377120  7097 consensus_queue.cc:260] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.379347  7097 ts_tablet_manager.cc:1434] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:27:52.379876  6725 catalog_manager.cc:5697] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:52.392169  6772 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:52.447547  7530 raft_consensus.cc:493] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.448081  7530 raft_consensus.cc:515] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:52.450423  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:52.449820  7530 leader_election.cc:290] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:52.450709  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:52.451130  6902 raft_consensus.cc:2468] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:52.451383  6975 raft_consensus.cc:2468] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 0.
I20260709 12:27:52.452157  6793 leader_election.cc:304] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:52.452930  7530 raft_consensus.cc:2804] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:52.453282  7530 raft_consensus.cc:493] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.453617  7530 raft_consensus.cc:3060] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.459214  7530 raft_consensus.cc:515] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:52.460734  7530 leader_election.cc:290] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:52.461277  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:52.461615  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:52.461791  6902 raft_consensus.cc:3060] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.462167  6975 raft_consensus.cc:3060] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.465036  7530 raft_consensus.cc:493] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.465544  7530 raft_consensus.cc:515] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:52.467206  6902 raft_consensus.cc:2468] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:52.468103  7530 leader_election.cc:290] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:52.468138  6975 raft_consensus.cc:2468] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 1.
I20260709 12:27:52.468369  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2b62ad0158424696dcdf73faa4f675" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:52.469075  6793 leader_election.cc:304] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:52.469556  6974 raft_consensus.cc:2393] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38b3b964f82644c7b654e8042953a74e in current term 1: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:52.468658  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e2b62ad0158424696dcdf73faa4f675" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:52.469919  7520 raft_consensus.cc:2804] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:52.470454  7520 raft_consensus.cc:697] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:27:52.471427  6791 leader_election.cc:304] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:52.471305  7520 consensus_queue.cc:237] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:52.472525  7530 raft_consensus.cc:3060] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.480278  7530 raft_consensus.cc:2749] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:52.481666  6725 catalog_manager.cc:5697] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 1 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.506485  7540 raft_consensus.cc:493] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.506964  7540 raft_consensus.cc:515] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.508739  7540 leader_election.cc:290] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:52.509289  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b065e90800f40379b501a6238c6fdcd" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:52.509666  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b065e90800f40379b501a6238c6fdcd" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:52.509974  7044 raft_consensus.cc:2468] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:52.510356  6824 raft_consensus.cc:2468] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 0.
I20260709 12:27:52.510979  6866 leader_election.cc:304] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:52.511682  7540 raft_consensus.cc:2804] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:52.511976  7540 raft_consensus.cc:493] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.512253  7540 raft_consensus.cc:3060] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.517057  7540 raft_consensus.cc:515] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.518795  7540 leader_election.cc:290] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:52.519505  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b065e90800f40379b501a6238c6fdcd" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:52.519728  6824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b065e90800f40379b501a6238c6fdcd" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:52.520085  7044 raft_consensus.cc:3060] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.520421  6824 raft_consensus.cc:3060] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:52.525604  7044 raft_consensus.cc:2468] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:52.525839  6824 raft_consensus.cc:2468] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:52.526628  6866 leader_election.cc:304] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:52.527371  7498 raft_consensus.cc:2804] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:52.527848  7498 raft_consensus.cc:697] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:52.528591  7498 consensus_queue.cc:237] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:52.539379  6725 catalog_manager.cc:5697] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:52.582327  7522 consensus_queue.cc:1048] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:52.595662  7522 consensus_queue.cc:1048] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:52.612249  7498 consensus_queue.cc:1048] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:52.624859  7540 consensus_queue.cc:1048] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:52.715169  7452 raft_consensus.cc:493] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fb7d0187b544dfdb8487ecb2ff24a76)
I20260709 12:27:52.715679  7452 raft_consensus.cc:515] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:52.717671  7452 leader_election.cc:290] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:52.718061  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:52.718943  6899 raft_consensus.cc:2393] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 2: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:52.718912  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:52.719583  7043 raft_consensus.cc:2468] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 2.
I20260709 12:27:52.720896  6940 leader_election.cc:304] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:52.722329  7497 raft_consensus.cc:2804] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:52.722702  7497 raft_consensus.cc:493] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fb7d0187b544dfdb8487ecb2ff24a76)
I20260709 12:27:52.723062  7497 raft_consensus.cc:3060] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.728497  7497 raft_consensus.cc:515] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:52.730456  7497 leader_election.cc:290] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:27:52.730674  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:52.730978  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864b9e85f77d4ecc8726f6d624b5780c" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:52.731542  6899 raft_consensus.cc:2393] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 2: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:52.737079  7043 raft_consensus.cc:2468] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 2.
I20260709 12:27:52.737989  6940 leader_election.cc:304] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:52.738713  7497 raft_consensus.cc:2804] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:52.739117  7497 raft_consensus.cc:697] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:52.739862  7497 consensus_queue.cc:237] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:27:52.750895  6722 catalog_manager.cc:5697] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 1 to 2, leader changed from 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68) to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 2 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.829530  7520 consensus_queue.cc:1048] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:52.840432  7530 consensus_queue.cc:1048] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:52.955610  7498 consensus_queue.cc:1048] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:52.964924  7466 consensus_queue.cc:1048] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:53.032626  7499 consensus_queue.cc:1048] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:53.042601  7520 consensus_queue.cc:1048] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:53.176786 29845 tablet_server.cc:179] TabletServer@127.29.37.65:0 shutting down...
W20260709 12:27:53.182451  7013 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:53.195478  7013 consensus_peers.cc:597] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.207620  6939 consensus_peers.cc:597] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.210281  7043 raft_consensus.cc:1275] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer c78bd8036f7040b99af7c501182cbe21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:27:53.210808  7013 consensus_peers.cc:597] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.212107  7575 consensus_queue.cc:1048] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:53.232436  6902 raft_consensus.cc:1275] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer c78bd8036f7040b99af7c501182cbe21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.235260  7575 consensus_queue.cc:1048] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:27:53.246440  6865 consensus_peers.cc:597] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.247931  7013 consensus_peers.cc:597] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.258147  7013 consensus_peers.cc:597] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.260459 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:53.262557 29845 tablet_replica.cc:333] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.263397 29845 raft_consensus.cc:2243] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.263888 29845 raft_consensus.cc:2272] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:53.264022  6865 consensus_peers.cc:597] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.266884 29845 tablet_replica.cc:333] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.267657 29845 raft_consensus.cc:2243] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.268126 29845 raft_consensus.cc:2272] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.271276 29845 tablet_replica.cc:333] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.271914 29845 raft_consensus.cc:2243] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.272786 29845 raft_consensus.cc:2272] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.275188 29845 tablet_replica.cc:333] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.275875 29845 raft_consensus.cc:2243] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.276690 29845 raft_consensus.cc:2272] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.278730 29845 tablet_replica.cc:333] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.279312 29845 raft_consensus.cc:2243] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.279695 29845 raft_consensus.cc:2272] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.281239 29845 tablet_replica.cc:333] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.281766 29845 raft_consensus.cc:2243] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.282135 29845 raft_consensus.cc:2272] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.283787 29845 tablet_replica.cc:333] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.284334 29845 raft_consensus.cc:2243] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.284993 29845 raft_consensus.cc:2272] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.286674 29845 tablet_replica.cc:333] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.287137 29845 raft_consensus.cc:2243] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.287568 29845 raft_consensus.cc:2272] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.289858 29845 tablet_replica.cc:333] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.290696 29845 raft_consensus.cc:2243] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.291172 29845 raft_consensus.cc:2272] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.292860 29845 tablet_replica.cc:333] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.293403 29845 raft_consensus.cc:2243] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.294035 29845 raft_consensus.cc:2272] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.295823 29845 tablet_replica.cc:333] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.296394 29845 raft_consensus.cc:2243] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.296770 29845 raft_consensus.cc:2272] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.298444 29845 tablet_replica.cc:333] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.298956 29845 raft_consensus.cc:2243] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.299443 29845 raft_consensus.cc:2272] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.301517 29845 tablet_replica.cc:333] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.302273 29845 raft_consensus.cc:2243] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.303130 29845 raft_consensus.cc:2272] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.305119 29845 tablet_replica.cc:333] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.305642 29845 raft_consensus.cc:2243] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.306115 29845 raft_consensus.cc:2272] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:53.307772  6865 consensus_peers.cc:597] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.307967 29845 tablet_replica.cc:333] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.308681 29845 raft_consensus.cc:2243] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.309122 29845 raft_consensus.cc:2272] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.310920 29845 tablet_replica.cc:333] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.311480 29845 raft_consensus.cc:2243] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.311867 29845 raft_consensus.cc:2272] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.313452 29845 tablet_replica.cc:333] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.314010 29845 raft_consensus.cc:2243] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.314711 29845 raft_consensus.cc:2272] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.316537 29845 tablet_replica.cc:333] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.317289 29845 raft_consensus.cc:2243] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.317756 29845 raft_consensus.cc:2272] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.319792 29845 tablet_replica.cc:333] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.320327 29845 raft_consensus.cc:2243] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.321053 29845 raft_consensus.cc:2272] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.323200 29845 tablet_replica.cc:333] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
W20260709 12:27:53.323789  6939 consensus_peers.cc:597] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.324038 29845 raft_consensus.cc:2243] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.325152 29845 raft_consensus.cc:2272] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.327283 29845 tablet_replica.cc:333] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.328181 29845 raft_consensus.cc:2243] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.328668 29845 raft_consensus.cc:2272] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.330659 29845 tablet_replica.cc:333] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.331209 29845 raft_consensus.cc:2243] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.331557 29845 raft_consensus.cc:2272] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.333317 29845 tablet_replica.cc:333] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.333963 29845 raft_consensus.cc:2243] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.334358 29845 raft_consensus.cc:2272] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.336015 29845 tablet_replica.cc:333] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.336503 29845 raft_consensus.cc:2243] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.336826 29845 raft_consensus.cc:2272] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.339203 29845 tablet_replica.cc:333] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.339936 29845 raft_consensus.cc:2243] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.340739 29845 raft_consensus.cc:2272] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.342427 29845 tablet_replica.cc:333] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.342924 29845 raft_consensus.cc:2243] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.343255 29845 raft_consensus.cc:2272] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.345628 29845 tablet_replica.cc:333] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.346309 29845 raft_consensus.cc:2243] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.347251 29845 raft_consensus.cc:2272] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.349131 29845 tablet_replica.cc:333] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.349766 29845 raft_consensus.cc:2243] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.350304 29845 raft_consensus.cc:2272] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.352555 29845 tablet_replica.cc:333] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.353102 29845 raft_consensus.cc:2243] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.353458 29845 raft_consensus.cc:2272] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.355360 29845 tablet_replica.cc:333] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.356112 29845 raft_consensus.cc:2243] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.356544 29845 raft_consensus.cc:2272] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:53.357646  6939 consensus_peers.cc:597] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.358700 29845 tablet_replica.cc:333] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
W20260709 12:27:53.358814  7013 consensus_peers.cc:597] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.359632 29845 raft_consensus.cc:2243] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.360189 29845 raft_consensus.cc:2272] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.362500 29845 tablet_replica.cc:333] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.363083 29845 raft_consensus.cc:2243] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.363492 29845 raft_consensus.cc:2272] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.365295 29845 tablet_replica.cc:333] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.365824 29845 raft_consensus.cc:2243] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.366247 29845 raft_consensus.cc:2272] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.367971 29845 tablet_replica.cc:333] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.368463 29845 raft_consensus.cc:2243] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.369174 29845 raft_consensus.cc:2272] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.370952 29845 tablet_replica.cc:333] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.371466 29845 raft_consensus.cc:2243] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.372242 29845 raft_consensus.cc:2272] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.374655 29845 tablet_replica.cc:333] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.375361 29845 raft_consensus.cc:2243] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.375829 29845 raft_consensus.cc:2272] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.377784 29845 tablet_replica.cc:333] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.378369 29845 raft_consensus.cc:2243] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.379083 29845 raft_consensus.cc:2272] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.380874 29845 tablet_replica.cc:333] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.381567 29845 raft_consensus.cc:2243] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.382058 29845 raft_consensus.cc:2272] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.384224 29845 tablet_replica.cc:333] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.384902 29845 raft_consensus.cc:2243] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.385784 29845 raft_consensus.cc:2272] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.388121 29845 tablet_replica.cc:333] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.388893 29845 raft_consensus.cc:2243] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.389430 29845 raft_consensus.cc:2272] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:53.390432  6939 consensus_peers.cc:597] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.391620 29845 tablet_replica.cc:333] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.392397 29845 raft_consensus.cc:2243] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.392872 29845 raft_consensus.cc:2272] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.395117 29845 tablet_replica.cc:333] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.395797 29845 raft_consensus.cc:2243] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.396703 29845 raft_consensus.cc:2272] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.398481 29845 tablet_replica.cc:333] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.398979 29845 raft_consensus.cc:2243] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.399685 29845 raft_consensus.cc:2272] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.401947 29845 tablet_replica.cc:333] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.402609 29845 raft_consensus.cc:2243] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:53.403510 29845 raft_consensus.cc:2272] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:53.405721 29845 tablet_replica.cc:333] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: stopping tablet replica
I20260709 12:27:53.406329 29845 raft_consensus.cc:2243] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:53.406805 29845 raft_consensus.cc:2272] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:53.437327  6865 consensus_peers.cc:597] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.458693  7013 consensus_peers.cc:597] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.472335  7013 consensus_peers.cc:597] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.477066 29845 tablet_server.cc:196] TabletServer@127.29.37.65:0 shutdown complete.
W20260709 12:27:53.493677  6939 consensus_peers.cc:597] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.507122  7013 consensus_peers.cc:597] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.507501  6939 consensus_peers.cc:597] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.515246  6939 consensus_peers.cc:597] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.536672 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:53.543597  7584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:53.543641  7583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:53.545666  7586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:53.546331 29845 server_base.cc:1061] running on GCE node
I20260709 12:27:53.551781 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:53.552179 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:53.552424 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600073552404 us; error 0 us; skew 500 ppm
I20260709 12:27:53.553248 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:53.560372 29845 webserver.cc:533] Webserver started at http://127.29.37.65:41905/ using document root <none> and password file <none>
I20260709 12:27:53.560986 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:53.561224 29845 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 12:27:53.562590  6865 consensus_peers.cc:597] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.565481 29845 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.000s	sys 0.002s
W20260709 12:27:53.569331  6939 consensus_peers.cc:597] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.573112  6939 consensus_peers.cc:597] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.575420  7592 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:53.576262 29845 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 12:27:53.576581 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "38b3b964f82644c7b654e8042953a74e"
format_stamp: "Formatted at 2026-07-09 12:27:42 on dist-test-slave-g1m6"
I20260709 12:27:53.576984 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:53.590061 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:53.591288 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:53.592974 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:53.597745  7599 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:27:53.631577  6939 consensus_peers.cc:597] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.636694  6865 consensus_peers.cc:597] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.637233  6865 consensus_peers.cc:597] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.645433  6865 consensus_peers.cc:597] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.664289  6865 consensus_peers.cc:597] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.681706  6865 consensus_peers.cc:597] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:53.709075  7013 consensus_peers.cc:597] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.835882 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:27:53.836274 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.240s	user 0.008s	sys 0.001s
I20260709 12:27:53.836634 29845 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:27:53.842140  7599 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:53.856710  7599 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:53.857609  7599 tablet_bootstrap.cc:492] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:53.858206  7599 ts_tablet_manager.cc:1403] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:27:53.860834  7599 raft_consensus.cc:359] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:53.861470  7599 raft_consensus.cc:740] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:53.862676  7599 consensus_queue.cc:260] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:53.870596  7599 ts_tablet_manager.cc:1434] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.008s
I20260709 12:27:53.871397  7599 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:53.889408  7599 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:53.890079  7599 tablet_bootstrap.cc:492] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:53.890602  7599 ts_tablet_manager.cc:1403] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:27:53.892488  7599 raft_consensus.cc:359] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:53.893375  7599 raft_consensus.cc:740] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:53.893898  7599 consensus_queue.cc:260] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:53.895187  7599 ts_tablet_manager.cc:1434] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:27:53.895841  7599 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:53.910997  7599 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:53.911676  7599 tablet_bootstrap.cc:492] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:53.912153  7599 ts_tablet_manager.cc:1403] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:27:53.913820  7599 raft_consensus.cc:359] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:53.914587  7599 raft_consensus.cc:740] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:53.915287  7599 consensus_queue.cc:260] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:53.917393  7599 ts_tablet_manager.cc:1434] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:53.918150  7599 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:53.965984  7599 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:53.967661  7599 tablet_bootstrap.cc:492] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:53.968313  7599 ts_tablet_manager.cc:1403] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.050s	user 0.028s	sys 0.016s
I20260709 12:27:53.970639  7599 raft_consensus.cc:359] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:53.971323  7599 raft_consensus.cc:740] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:53.972029  7599 consensus_queue.cc:260] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:53.973726  7599 ts_tablet_manager.cc:1434] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:53.974466  7599 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.012507  7599 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.014000  7599 tablet_bootstrap.cc:492] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.014673  7599 ts_tablet_manager.cc:1403] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.019s
I20260709 12:27:54.017096  7599 raft_consensus.cc:359] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.017779  7599 raft_consensus.cc:740] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.018446  7599 consensus_queue.cc:260] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.020373  7599 ts_tablet_manager.cc:1434] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:54.020988  7599 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.037035  7599 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.038288  7599 tablet_bootstrap.cc:492] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.038947  7599 ts_tablet_manager.cc:1403] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 12:27:54.041111  7599 raft_consensus.cc:359] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.041751  7599 raft_consensus.cc:740] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.042423  7599 consensus_queue.cc:260] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.044433  7599 ts_tablet_manager.cc:1434] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:54.045156  7599 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.058528  7599 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.059419  7599 tablet_bootstrap.cc:492] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.060024  7599 ts_tablet_manager.cc:1403] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 12:27:54.062194  7599 raft_consensus.cc:359] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.062856  7599 raft_consensus.cc:740] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.065912  7599 consensus_queue.cc:260] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.068054  7599 ts_tablet_manager.cc:1434] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:27:54.068754  7599 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.081853  7599 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.082724  7599 tablet_bootstrap.cc:492] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.083321  7599 ts_tablet_manager.cc:1403] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:27:54.085494  7599 raft_consensus.cc:359] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.086134  7599 raft_consensus.cc:740] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.086773  7599 consensus_queue.cc:260] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.088687  7599 ts_tablet_manager.cc:1434] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:54.089447  7599 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.100175 29845 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:27:54.100641 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.264s	user 0.230s	sys 0.006s
I20260709 12:27:54.107568  7599 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.108469  7599 tablet_bootstrap.cc:492] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.109061  7599 ts_tablet_manager.cc:1403] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 12:27:54.111796  7599 raft_consensus.cc:359] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.112703  7599 raft_consensus.cc:740] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.113471  7599 consensus_queue.cc:260] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.116320  7599 ts_tablet_manager.cc:1434] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 12:27:54.117327  7599 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.139468  7599 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.140219  7599 tablet_bootstrap.cc:492] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.140779  7599 ts_tablet_manager.cc:1403] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 12:27:54.144385  7599 raft_consensus.cc:359] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.145253  7599 raft_consensus.cc:740] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.146334  7599 consensus_queue.cc:260] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.149354  7599 ts_tablet_manager.cc:1434] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:27:54.150205  7599 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.178340  7599 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.179502  7599 tablet_bootstrap.cc:492] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.180259  7599 ts_tablet_manager.cc:1403] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.000s
I20260709 12:27:54.183156  7599 raft_consensus.cc:359] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.183995  7599 raft_consensus.cc:740] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.184793  7599 consensus_queue.cc:260] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.187284  7599 ts_tablet_manager.cc:1434] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:27:54.188207  7599 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.204721  7599 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.205631  7599 tablet_bootstrap.cc:492] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.206147  7599 ts_tablet_manager.cc:1403] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 12:27:54.209784  7599 raft_consensus.cc:359] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.210470  7599 raft_consensus.cc:740] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.211189  7599 consensus_queue.cc:260] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.213284  7599 ts_tablet_manager.cc:1434] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:27:54.214017  7599 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.234539  7599 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.235524  7599 tablet_bootstrap.cc:492] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.236356  7599 ts_tablet_manager.cc:1403] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 12:27:54.242259  7599 raft_consensus.cc:359] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.242944  7599 raft_consensus.cc:740] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.243642  7599 consensus_queue.cc:260] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.245589  7599 ts_tablet_manager.cc:1434] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:27:54.251013  7599 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.256314 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:45171
I20260709 12:27:54.256449  7671 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:45171 every 8 connection(s)
I20260709 12:27:54.269667  7599 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.270869  7599 tablet_bootstrap.cc:492] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.271517  7599 ts_tablet_manager.cc:1403] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 12:27:54.273973  7599 raft_consensus.cc:359] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.274525  7599 raft_consensus.cc:740] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.275236  7599 consensus_queue.cc:260] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.277035  7599 ts_tablet_manager.cc:1434] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:54.277717  7599 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.289479  7672 heartbeater.cc:344] Connected to a master server at 127.29.37.126:36231
I20260709 12:27:54.289994  7672 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:54.290894  7672 heartbeater.cc:507] Master 127.29.37.126:36231 requested a full tablet report, sending...
I20260709 12:27:54.293128  7599 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.294118  7599 tablet_bootstrap.cc:492] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.294771  7599 ts_tablet_manager.cc:1403] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.005s
I20260709 12:27:54.297122  7599 raft_consensus.cc:359] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.297771  7599 raft_consensus.cc:740] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.298421  7599 consensus_queue.cc:260] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.300149  7599 ts_tablet_manager.cc:1434] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 12:27:54.300064  7466 raft_consensus.cc:493] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.301110  7599 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.300720  7466 raft_consensus.cc:515] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.302929  7466 leader_election.cc:290] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:54.304169  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.308285  7496 raft_consensus.cc:493] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.308789  7496 raft_consensus.cc:515] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.310935  7496 leader_election.cc:290] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.312825  6725 ts_manager.cc:194] Re-registered known tserver with Master: 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.313268  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.314610  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.322644  6902 raft_consensus.cc:2468] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:54.318981  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.323347  7599 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:27:54.323635  6939 leader_election.cc:343] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.324291  7599 tablet_bootstrap.cc:492] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
W20260709 12:27:54.324385  6865 leader_election.cc:343] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.324847  6865 leader_election.cc:304] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:54.325047  7599 ts_tablet_manager.cc:1403] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.015s
I20260709 12:27:54.325762  7571 raft_consensus.cc:2749] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.327423  7599 raft_consensus.cc:359] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.327962  7599 raft_consensus.cc:740] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.328526  7599 consensus_queue.cc:260] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.329877  7599 ts_tablet_manager.cc:1434] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:27:54.330550  7599 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.334510  6942 leader_election.cc:304] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.337284  7496 raft_consensus.cc:2804] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.337684  7496 raft_consensus.cc:493] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.338019  7496 raft_consensus.cc:3060] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.339879  7571 raft_consensus.cc:493] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.340261  7575 raft_consensus.cc:493] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.340410  7571 raft_consensus.cc:515] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.340804  7575 raft_consensus.cc:515] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.343420  7575 leader_election.cc:290] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.344463  7600 raft_consensus.cc:493] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.345171  7600 raft_consensus.cc:515] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.345762  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:54.349860  6939 leader_election.cc:343] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.350382  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.351158  6902 raft_consensus.cc:2468] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.352257  7600 leader_election.cc:290] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.353379  7016 leader_election.cc:304] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.355477  7522 raft_consensus.cc:2804] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.355962  7522 raft_consensus.cc:493] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.356338  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.356424  7522 raft_consensus.cc:3060] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.357084  7645 raft_consensus.cc:2468] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.360028  7452 raft_consensus.cc:493] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.361590  7452 raft_consensus.cc:515] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.362409  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.369352  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.370159  7645 raft_consensus.cc:2468] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:54.373822  6865 leader_election.cc:304] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.375488  7540 raft_consensus.cc:2804] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.375859  7540 raft_consensus.cc:493] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.376201  7540 raft_consensus.cc:3060] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.379446  6942 leader_election.cc:304] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:54.380556  7575 raft_consensus.cc:2749] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.383935  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.386902  7466 raft_consensus.cc:493] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.387804  7466 raft_consensus.cc:515] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.389065  7571 leader_election.cc:290] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.398190  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.399919  7540 raft_consensus.cc:515] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.403631  6725 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49802
I20260709 12:27:54.403857  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:54.405145  6939 leader_election.cc:343] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.405540  6939 leader_election.cc:304] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:54.405745  7540 leader_election.cc:290] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.406345  7575 raft_consensus.cc:2749] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.406986  7452 leader_election.cc:290] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.408080  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.408704  7644 raft_consensus.cc:3060] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.409737  7672 heartbeater.cc:499] Master 127.29.37.126:36231 was elected leader, sending a full tablet report...
I20260709 12:27:54.412354  7466 leader_election.cc:290] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.415365  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:54.416605  6865 leader_election.cc:343] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.397751  7522 raft_consensus.cc:515] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.423195  7047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:54.423612  7044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.423683  7600 raft_consensus.cc:493] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.420435  7644 raft_consensus.cc:2468] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 2.
I20260709 12:27:54.420044  7496 raft_consensus.cc:515] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.424477  7600 raft_consensus.cc:515] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.425773  7043 raft_consensus.cc:2393] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99db41e57175426b9ceb11625b4455d9 in current term 2: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:54.428961  7600 leader_election.cc:290] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.430234  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.430446  7601 raft_consensus.cc:493] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.431200  7601 raft_consensus.cc:515] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.433760  7601 leader_election.cc:290] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.434852  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
W20260709 12:27:54.436777  6939 leader_election.cc:343] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.437192  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.436780  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.440900  7452 raft_consensus.cc:493] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.441553  7452 raft_consensus.cc:515] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.446403  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:54.447988  6939 leader_election.cc:343] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.448825  7452 leader_election.cc:290] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.451161  7496 leader_election.cc:290] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.454994  7571 raft_consensus.cc:493] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.455698  7571 raft_consensus.cc:515] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.462311  7571 leader_election.cc:290] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.470705  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:54.471774  6902 raft_consensus.cc:2393] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 2: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:54.475353  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.479458  6865 leader_election.cc:304] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.481251  7571 raft_consensus.cc:2804] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.481891  7571 raft_consensus.cc:697] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:54.483579  7571 consensus_queue.cc:237] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.484979  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.480098  7522 leader_election.cc:290] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
W20260709 12:27:54.488926  6865 leader_election.cc:343] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:54.491670  7013 leader_election.cc:343] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.492134  7013 leader_election.cc:304] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
W20260709 12:27:54.493165  7013 leader_election.cc:343] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.493518  7013 leader_election.cc:304] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:54.495178  7682 raft_consensus.cc:2749] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.495385  7600 raft_consensus.cc:2749] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.512303  7599 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.513235  7599 tablet_bootstrap.cc:492] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.513854  7599 ts_tablet_manager.cc:1403] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.183s	user 0.073s	sys 0.082s
I20260709 12:27:54.515712  7683 raft_consensus.cc:493] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.516208  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f932aaa29a8443d4b251dea7c9fc0f54" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.516501  7599 raft_consensus.cc:359] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.517045  7645 raft_consensus.cc:2393] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 2: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:54.516664  7683 raft_consensus.cc:515] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.517284  7599 raft_consensus.cc:740] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
W20260709 12:27:54.515615  6866 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 85429.2us
I20260709 12:27:54.517994  7599 consensus_queue.cc:260] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.518806  6866 leader_election.cc:304] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:54.519562  7540 raft_consensus.cc:2749] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.520300  7599 ts_tablet_manager.cc:1434] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.009s	sys 0.000s
I20260709 12:27:54.521093  7599 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.523233  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.523898  7644 raft_consensus.cc:2468] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:54.524355  7683 leader_election.cc:290] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.524987  6865 leader_election.cc:304] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.525905  7013 leader_election.cc:304] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:54.530117  7540 raft_consensus.cc:2804] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.530400  7600 raft_consensus.cc:2749] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.530522  7540 raft_consensus.cc:493] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.531183  7540 raft_consensus.cc:3060] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.532994  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.539155  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.542348  7599 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.543310  7599 tablet_bootstrap.cc:492] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.543503  7681 raft_consensus.cc:493] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.543915  7599 ts_tablet_manager.cc:1403] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.001s
I20260709 12:27:54.544013  7681 raft_consensus.cc:515] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.545747  7575 raft_consensus.cc:493] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.544062  7466 raft_consensus.cc:493] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.546296  7575 raft_consensus.cc:515] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.546803  7599 raft_consensus.cc:359] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.546607  6722 catalog_manager.cc:5697] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.547629  7599 raft_consensus.cc:740] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.548352  7599 consensus_queue.cc:260] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.548322  7575 leader_election.cc:290] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.546954  7466 raft_consensus.cc:515] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.549955  7571 raft_consensus.cc:493] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.550693  7599 ts_tablet_manager.cc:1434] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:27:54.550477  7571 raft_consensus.cc:515] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.551354  7466 leader_election.cc:290] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:54.551483  7599 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.552416  7571 leader_election.cc:290] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:54.552721  7642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.554807  7681 leader_election.cc:290] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:54.554980  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:54.557462  6939 leader_election.cc:343] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.561828  7600 raft_consensus.cc:493] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.562355  7600 raft_consensus.cc:515] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.562747  7540 raft_consensus.cc:515] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.564471  7600 leader_election.cc:290] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.564776  7540 leader_election.cc:290] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.564846  7683 raft_consensus.cc:493] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.565515  7683 raft_consensus.cc:515] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.566372  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.567301  7683 leader_election.cc:290] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.570577  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.570570  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.571507  6902 raft_consensus.cc:2468] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
W20260709 12:27:54.572445  7013 leader_election.cc:343] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.573283  7016 leader_election.cc:304] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.567232  6897 raft_consensus.cc:2468] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:54.575335  7600 raft_consensus.cc:2804] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.575867  7600 raft_consensus.cc:493] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.575824  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ac46831fcf945d194fd7677ada23856" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:54.576967  6899 raft_consensus.cc:3060] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.579778  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.580662  6898 raft_consensus.cc:2468] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:54.585331  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:54.587241  7600 raft_consensus.cc:3060] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.595630  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.596516  7043 raft_consensus.cc:2468] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:54.598703  6942 leader_election.cc:304] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.600438  6942 leader_election.cc:304] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.602057  7575 raft_consensus.cc:2804] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.603229  7575 raft_consensus.cc:493] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.603590  7575 raft_consensus.cc:3060] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:27:54.604801  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 55937.6us
I20260709 12:27:54.602172  7496 raft_consensus.cc:2804] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.606644  7496 raft_consensus.cc:493] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.607030  7496 raft_consensus.cc:3060] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.608022  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.609505  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.610760  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.611881  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.612447  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.612488  6974 raft_consensus.cc:2468] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:54.615509  6866 leader_election.cc:304] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
W20260709 12:27:54.615867  6865 leader_election.cc:343] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.617652  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.616918  7466 raft_consensus.cc:2749] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.618721  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.620925  7644 raft_consensus.cc:3060] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.607048  6899 raft_consensus.cc:2468] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 2.
I20260709 12:27:54.607128  7600 raft_consensus.cc:515] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.624804  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
W20260709 12:27:54.626130  7013 leader_election.cc:343] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.628086  7644 raft_consensus.cc:2468] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 2.
I20260709 12:27:54.631024  7600 leader_election.cc:290] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.625277  7575 raft_consensus.cc:515] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.632117  6866 leader_election.cc:304] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.633071  6973 raft_consensus.cc:2393] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99db41e57175426b9ceb11625b4455d9 in current term 2: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:54.633944  7683 raft_consensus.cc:2804] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.634377  7683 raft_consensus.cc:493] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
W20260709 12:27:54.629863  6865 leader_election.cc:343] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.634889  7683 raft_consensus.cc:3060] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.635193  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e11f6b3dd5fe466d984f796948818c0d" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:54.635949  6899 raft_consensus.cc:3060] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.635272  7575 leader_election.cc:290] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.634591  7689 raft_consensus.cc:493] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.637018  7689 raft_consensus.cc:515] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.637640  6942 leader_election.cc:304] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.638938  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.639724  7452 raft_consensus.cc:2804] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.640018  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.640250  7452 raft_consensus.cc:697] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:54.641204  7452 consensus_queue.cc:237] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
W20260709 12:27:54.642320  6865 leader_election.cc:343] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:54.643260  6939 leader_election.cc:343] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.646283  7599 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.641909  7522 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
W20260709 12:27:54.646718  6865 leader_election.cc:343] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.646879  7522 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
W20260709 12:27:54.646283  6939 leader_election.cc:343] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.649507  7599 tablet_bootstrap.cc:492] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.649955  7522 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.652122  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.650023  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0aa7e52e7c5b4039a78f136142693567" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:54.663398  7599 ts_tablet_manager.cc:1403] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.112s	user 0.010s	sys 0.018s
I20260709 12:27:54.667421  6902 raft_consensus.cc:3060] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.667487  7689 leader_election.cc:290] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.640033  7496 raft_consensus.cc:515] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.668604  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.669831  6975 raft_consensus.cc:2393] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99db41e57175426b9ceb11625b4455d9 in current term 2: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:54.670576  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.651010  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.671741  7643 raft_consensus.cc:2468] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.672581  6942 leader_election.cc:304] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:54.672971  7013 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: c78bd8036f7040b99af7c501182cbe21
W20260709 12:27:54.671804  6939 leader_election.cc:343] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.674293  7575 raft_consensus.cc:2749] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.674384  7600 raft_consensus.cc:2804] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.674918  7600 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.675292  7600 raft_consensus.cc:3060] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.675448  6865 leader_election.cc:304] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:54.676790  7599 raft_consensus.cc:359] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.678767  7599 raft_consensus.cc:740] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.679188  7496 leader_election.cc:290] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.679277  6902 raft_consensus.cc:2468] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 2.
I20260709 12:27:54.647748  7683 raft_consensus.cc:515] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.679939  7599 consensus_queue.cc:260] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.678297  6899 raft_consensus.cc:2468] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:27:54.681917  7683 leader_election.cc:290] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.682186  7016 leader_election.cc:304] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.682557  7466 raft_consensus.cc:2804] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.683213  7466 raft_consensus.cc:697] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:54.682476  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:54.684046  7466 consensus_queue.cc:237] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.684490  7522 raft_consensus.cc:2804] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.685247  7599 ts_tablet_manager.cc:1434] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:27:54.685564  7522 raft_consensus.cc:697] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:54.684497  7043 raft_consensus.cc:3060] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.686396  7599 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.686513  7522 consensus_queue.cc:237] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.695564  7600 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.697099  7043 raft_consensus.cc:2468] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 2.
I20260709 12:27:54.698112  6866 leader_election.cc:304] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.696611  6942 leader_election.cc:304] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.702247  7466 raft_consensus.cc:2804] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.702941  7689 raft_consensus.cc:2804] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.703207  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a690e13ef48d439cb389b897c6b9fcc6" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:54.703877  6865 leader_election.cc:304] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.704006  7689 raft_consensus.cc:697] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:54.704694  7540 raft_consensus.cc:2804] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.705116  7540 raft_consensus.cc:493] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.705485  7540 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.705395  7689 consensus_queue.cc:237] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.704320  7466 raft_consensus.cc:697] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:54.708155  7496 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.708438  7466 consensus_queue.cc:237] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.708815  7496 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.704032  6902 raft_consensus.cc:3060] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.711957  7601 raft_consensus.cc:493] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.712635  7601 raft_consensus.cc:515] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.731942  6865 leader_election.cc:304] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:54.738684  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.739336  7496 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.739557  7043 raft_consensus.cc:2393] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 2: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:54.739710  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.740337  7643 raft_consensus.cc:2468] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:54.741222  6865 leader_election.cc:304] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:54.741657  6939 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:54.742463  7694 raft_consensus.cc:2804] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.734308  7683 raft_consensus.cc:2749] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.740337  7452 raft_consensus.cc:493] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.743923  7452 raft_consensus.cc:515] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.742945  7694 raft_consensus.cc:493] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.745160  7694 raft_consensus.cc:3060] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.747465  6724 catalog_manager.cc:5697] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 2 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:54.750522  6942 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 54000.3us
I20260709 12:27:54.751331  7522 raft_consensus.cc:493] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.753098  7452 leader_election.cc:290] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.754263  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:54.755688  6939 leader_election.cc:343] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.755946  7601 leader_election.cc:290] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.756431  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.757730  6940 leader_election.cc:304] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:54.757884  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.758646  7644 raft_consensus.cc:3060] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.760103  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.760950  7643 raft_consensus.cc:2393] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 2: Already voted for candidate 99db41e57175426b9ceb11625b4455d9 in this term.
I20260709 12:27:54.760962  7691 raft_consensus.cc:2749] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.757872  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2429fdc971b4c048fbc5f3ce3b6bc2f" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.751824  7522 raft_consensus.cc:515] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.764186  6722 catalog_manager.cc:5697] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 2 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:54.765549  7016 leader_election.cc:304] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:27:54.766685  7644 raft_consensus.cc:2468] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:27:54.767869  7682 raft_consensus.cc:3060] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.767812  7013 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:54.771334  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:54.774446  7601 raft_consensus.cc:2804] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.775105  7522 leader_election.cc:290] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.775099  7683 raft_consensus.cc:493] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.775864  7683 raft_consensus.cc:515] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.776618  7686 raft_consensus.cc:493] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.777356  7600 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.777189  7686 raft_consensus.cc:515] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.777801  7683 leader_election.cc:290] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:54.780409  7686 leader_election.cc:290] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.777030  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.781013  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ba9e3e56242779781f03654f3dfd3" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.781980  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.782552  6902 raft_consensus.cc:2468] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 2.
I20260709 12:27:54.782693  7682 raft_consensus.cc:2749] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.795064  7681 raft_consensus.cc:2749] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.797497  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.787689  7601 raft_consensus.cc:697] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:54.793527  7600 raft_consensus.cc:493] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.802007  6973 raft_consensus.cc:2468] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.808151  7599 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:54.809103  7599 tablet_bootstrap.cc:492] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:54.808736  7601 consensus_queue.cc:237] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.810194  7600 raft_consensus.cc:515] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.812415  7600 leader_election.cc:290] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.809803  7599 ts_tablet_manager.cc:1403] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.124s	user 0.025s	sys 0.025s
I20260709 12:27:54.813004  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.813575  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.814177  6902 raft_consensus.cc:2468] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
W20260709 12:27:54.814828  7013 leader_election.cc:343] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.815358  7016 leader_election.cc:304] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.817919  6942 leader_election.cc:304] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.819490  7013 leader_election.cc:304] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:54.819715  7599 raft_consensus.cc:359] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.820580  7599 raft_consensus.cc:740] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:54.821503  7599 consensus_queue.cc:260] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.820334  7600 raft_consensus.cc:2804] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.826738  7640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.822192  7682 raft_consensus.cc:493] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.826848  7600 raft_consensus.cc:493] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.826894  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.822036  7689 raft_consensus.cc:2804] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.828512  7645 raft_consensus.cc:2468] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:54.828680  7689 raft_consensus.cc:697] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:54.827653  7682 raft_consensus.cc:515] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.829344  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.829991  6973 raft_consensus.cc:2468] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:54.831248  7682 leader_election.cc:290] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.832202  7691 raft_consensus.cc:493] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.832722  7691 raft_consensus.cc:515] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.827833  7600 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.836037  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.836676  7691 leader_election.cc:290] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.836719  7643 raft_consensus.cc:2468] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 1.
I20260709 12:27:54.838140  6939 leader_election.cc:304] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:54.838970  7691 raft_consensus.cc:2804] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.839498  7691 raft_consensus.cc:493] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.839628  7047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.839972  7691 raft_consensus.cc:3060] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.840889  7683 raft_consensus.cc:493] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.829620  7689 consensus_queue.cc:237] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.841456  7683 raft_consensus.cc:515] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.843501  7683 leader_election.cc:290] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.839977  6865 leader_election.cc:304] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:54.800536  7644 raft_consensus.cc:2468] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.847780  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.848407  7047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" is_pre_election: true
I20260709 12:27:54.849038  7047 raft_consensus.cc:2468] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 1.
I20260709 12:27:54.849854  7683 raft_consensus.cc:2804] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.850275  7683 raft_consensus.cc:493] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.850618  7683 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.851902  6866 leader_election.cc:304] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.853279  7682 raft_consensus.cc:493] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.853773  7682 raft_consensus.cc:515] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:27:54.846726  6865 leader_election.cc:343] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:54.855279  6865 leader_election.cc:343] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:54.855366  7013 leader_election.cc:343] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:54.824764  7700 raft_consensus.cc:2804] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.859660  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.859934  7700 raft_consensus.cc:493] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.857966  7681 raft_consensus.cc:2804] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.860371  6973 raft_consensus.cc:2468] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.860939  7681 raft_consensus.cc:493] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.861346  7681 raft_consensus.cc:3060] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.862670  7522 raft_consensus.cc:493] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.863179  7522 raft_consensus.cc:515] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.864014  7682 leader_election.cc:290] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.865059  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:27:54.865835  6899 raft_consensus.cc:2468] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.867620  7016 leader_election.cc:304] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.865082  7522 leader_election.cc:290] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.869149  7522 raft_consensus.cc:2804] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.869496  7522 raft_consensus.cc:493] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.869848  7522 raft_consensus.cc:3060] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.860394  7700 raft_consensus.cc:3060] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.855295  7599 ts_tablet_manager.cc:1434] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 12:27:54.879324  7599 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:54.870762  7694 raft_consensus.cc:515] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.892441  7681 raft_consensus.cc:515] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.883692  7013 leader_election.cc:304] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:54.884626  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.893455  6975 raft_consensus.cc:2393] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 2: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:54.893761  7644 raft_consensus.cc:2468] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.885185  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:27:54.859184  6724 catalog_manager.cc:5697] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 2 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.894747  7682 raft_consensus.cc:2804] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.895232  7682 raft_consensus.cc:493] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.894821  7013 leader_election.cc:304] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:54.895637  7682 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.886096  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
I20260709 12:27:54.897079  7643 raft_consensus.cc:2468] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 1.
I20260709 12:27:54.897245  7540 raft_consensus.cc:515] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.899016  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:54.899767  7681 leader_election.cc:290] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.901151  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:54.893071  7691 raft_consensus.cc:515] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.901743  7043 raft_consensus.cc:3060] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.902354  6973 raft_consensus.cc:2393] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 2: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:54.904318  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e9253071944b29a6116b8d9f64ef3a" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.907440  7643 raft_consensus.cc:2393] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 2: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:54.907948  7047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:27:54.909034  7694 leader_election.cc:290] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.906024  7705 raft_consensus.cc:2804] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:54.906324  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "c78bd8036f7040b99af7c501182cbe21" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.910166  7644 raft_consensus.cc:3060] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.892236  7683 raft_consensus.cc:515] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:54.908975  6939 leader_election.cc:304] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:54.912250  7683 leader_election.cc:290] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:54.915818  7452 raft_consensus.cc:2749] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:27:54.904791  7691 leader_election.cc:290] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.926726  7705 raft_consensus.cc:493] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:27:54.927335  7705 raft_consensus.cc:3060] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.926072  7644 raft_consensus.cc:2468] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c78bd8036f7040b99af7c501182cbe21 in term 2.
I20260709 12:27:54.929409  6939 leader_election.cc:304] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:54.930280  7691 raft_consensus.cc:2804] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.933050  7682 raft_consensus.cc:515] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.935737  7682 leader_election.cc:290] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.936759  7700 raft_consensus.cc:515] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.930786  7691 raft_consensus.cc:697] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Becoming Leader. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:27:54.939235  7700 leader_election.cc:290] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.921978  7522 raft_consensus.cc:515] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.941314  7691 consensus_queue.cc:237] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.942755  7522 leader_election.cc:290] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:54.942889  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:54.945302  6902 raft_consensus.cc:3060] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.946293  7600 raft_consensus.cc:515] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.952840  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:27:54.953558  6899 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.960610  7705 raft_consensus.cc:515] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:54.964510  7047 raft_consensus.cc:2393] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c78bd8036f7040b99af7c501182cbe21 in current term 2: Already voted for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in this term.
I20260709 12:27:54.964524  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:54.965245  6973 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.961318  6899 raft_consensus.cc:2468] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:27:54.966266  6902 raft_consensus.cc:2468] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:27:54.967697  7016 leader_election.cc:304] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.967955  7705 leader_election.cc:290] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:27:54.971825  7016 leader_election.cc:304] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.972155  6973 raft_consensus.cc:2468] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:27:54.972764  7522 raft_consensus.cc:2804] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.973217  7522 raft_consensus.cc:697] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:54.973872  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.974473  7047 raft_consensus.cc:1275] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 99db41e57175426b9ceb11625b4455d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:54.945734  7043 raft_consensus.cc:2468] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 2.
I20260709 12:27:54.856397  6865 leader_election.cc:304] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:27:54.976169  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.976763  7683 raft_consensus.cc:2764] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:27:54.976897  7644 raft_consensus.cc:3060] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:27:54.977838  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 121500us
I20260709 12:27:54.978785  7708 raft_consensus.cc:2804] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.974063  7522 consensus_queue.cc:237] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.979480  7708 raft_consensus.cc:697] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:54.983448  6866 leader_election.cc:304] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:27:54.984331  7708 consensus_queue.cc:237] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.984778  7681 consensus_queue.cc:1048] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:54.985234  7686 raft_consensus.cc:2804] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:54.985679  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8203d585d06840daadbcfbb32fb2f822" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:54.985791  7686 raft_consensus.cc:697] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:54.986331  6973 raft_consensus.cc:3060] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.986771  7686 consensus_queue.cc:237] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:54.990110  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b8fc940ac6a48f0ab40f16b9a7b9ca6" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:54.990732  7643 raft_consensus.cc:3060] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:54.992684  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:54.999608  7640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7eee84ade114ccaa536076ae9b04ac2" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:55.000588  7640 raft_consensus.cc:2393] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in current term 2: Already voted for candidate c78bd8036f7040b99af7c501182cbe21 in this term.
I20260709 12:27:55.006589  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
W20260709 12:27:55.008265  7013 leader_election.cc:343] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:55.011814  7013 leader_election.cc:343] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:55.012441  7013 leader_election.cc:304] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:55.013569  7013 leader_election.cc:304] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:27:55.013646  7700 raft_consensus.cc:2804] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:55.014281  7522 raft_consensus.cc:2749] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:27:55.014432  7700 raft_consensus.cc:697] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:54.985001  7644 raft_consensus.cc:2468] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:27:55.015764  7700 consensus_queue.cc:237] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:55.020390  7600 leader_election.cc:290] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:27:55.014392  7643 raft_consensus.cc:2468] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
W20260709 12:27:55.024901  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 46417us
I20260709 12:27:55.026290  7013 leader_election.cc:304] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:27:55.034536  7522 raft_consensus.cc:2804] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:55.035050  7522 raft_consensus.cc:697] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:27:55.035063  7645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:55.035377  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:55.035993  7522 consensus_queue.cc:237] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:55.036832  6973 raft_consensus.cc:2468] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:27:54.984390  7540 leader_election.cc:290] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Requested vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:27:55.043884  7643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c55f4a15fa641de92c4922bc65c06be" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e" is_pre_election: true
W20260709 12:27:55.110710  6865 leader_election.cc:343] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:55.112403  6865 leader_election.cc:343] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:55.114861  6865 leader_election.cc:343] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:55.124275  7644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38b3b964f82644c7b654e8042953a74e"
I20260709 12:27:55.131780  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:55.132972  6973 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:27:55.135702  6939 consensus_peers.cc:597] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.137331  6902 raft_consensus.cc:1275] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer c78bd8036f7040b99af7c501182cbe21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.140038  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "99db41e57175426b9ceb11625b4455d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:27:55.140978  6975 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:55.147396  6899 raft_consensus.cc:1275] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.151494  7708 consensus_queue.cc:1048] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.156656  7599 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:55.158054  7599 tablet_bootstrap.cc:492] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:55.161307  7691 consensus_queue.cc:1048] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.181960  6975 raft_consensus.cc:2468] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 2.
I20260709 12:27:55.192539  6724 catalog_manager.cc:5697] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:55.199263  7644 raft_consensus.cc:1275] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Refusing update from remote peer 99db41e57175426b9ceb11625b4455d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.199393  7642 raft_consensus.cc:1275] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Refusing update from remote peer 99db41e57175426b9ceb11625b4455d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.190006  6721 catalog_manager.cc:5697] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 2 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:55.197208  6724 catalog_manager.cc:5697] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:55.207532  6725 catalog_manager.cc:5697] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 2 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:55.208808  7013 consensus_peers.cc:597] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.246343  7599 ts_tablet_manager.cc:1403] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.367s	user 0.034s	sys 0.040s
I20260709 12:27:55.253476  6865 leader_election.cc:304] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:27:55.261516  7044 raft_consensus.cc:1275] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 99db41e57175426b9ceb11625b4455d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.263767  7683 consensus_queue.cc:1048] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.273586  6898 raft_consensus.cc:1275] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer c78bd8036f7040b99af7c501182cbe21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.283849  6972 raft_consensus.cc:1275] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.287364  7689 consensus_queue.cc:1048] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.287570  7705 consensus_queue.cc:1048] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:27:55.290791  6865 leader_election.cc:343] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:55.291088  7599 raft_consensus.cc:359] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:55.292080  7599 raft_consensus.cc:740] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:55.294196  7599 consensus_queue.cc:260] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
W20260709 12:27:55.295318  6939 consensus_peers.cc:597] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.297909  7599 ts_tablet_manager.cc:1434] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.004s
I20260709 12:27:55.298321  6973 raft_consensus.cc:2468] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99db41e57175426b9ceb11625b4455d9 in term 2.
I20260709 12:27:55.291908  7683 raft_consensus.cc:2804] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:55.299407  7599 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:55.307935  7571 consensus_queue.cc:1048] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.292204  7645 raft_consensus.cc:1275] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.309653  7540 consensus_queue.cc:1048] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:27:55.315536  7708 consensus_queue.cc:1048] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:27:55.336792  6939 consensus_peers.cc:597] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.325546  7683 raft_consensus.cc:697] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:55.363236  6865 leader_election.cc:304] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21; no voters: 38b3b964f82644c7b654e8042953a74e
I20260709 12:27:55.364761  7540 raft_consensus.cc:2804] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:55.365502  7540 raft_consensus.cc:697] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Becoming Leader. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:27:55.366820  7540 consensus_queue.cc:237] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
W20260709 12:27:55.386983  7013 consensus_peers.cc:597] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.420076  7720 log.cc:927] Time spent T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:27:55.430392  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 67337.3us
W20260709 12:27:55.423370  6942 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 74807us
I20260709 12:27:55.425971  6725 catalog_manager.cc:5697] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:55.455070  6721 catalog_manager.cc:5697] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 2 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:55.462880  6721 catalog_manager.cc:5697] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 2 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:55.464226  6724 catalog_manager.cc:5697] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to c78bd8036f7040b99af7c501182cbe21 (127.29.37.67). New cstate: current_term: 2 leader_uuid: "c78bd8036f7040b99af7c501182cbe21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:55.466687  7642 raft_consensus.cc:1275] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:27:55.476445  7013 consensus_peers.cc:597] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.474754  6721 catalog_manager.cc:5697] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 2 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:55.487811  6721 catalog_manager.cc:5697] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 2 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:55.500407  6898 raft_consensus.cc:1275] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer c78bd8036f7040b99af7c501182cbe21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.489619  7683 consensus_queue.cc:237] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:55.534724  6902 raft_consensus.cc:1275] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.539397  7716 consensus_queue.cc:1048] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 12:27:55.576475  7645 raft_consensus.cc:1275] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.585511  7645 raft_consensus.cc:1275] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Refusing update from remote peer c78bd8036f7040b99af7c501182cbe21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.589193  7044 raft_consensus.cc:1275] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer c78bd8036f7040b99af7c501182cbe21: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.596660  7689 consensus_queue.cc:1048] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:27:55.604496  7691 consensus_queue.cc:1048] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.618695  7522 consensus_queue.cc:1048] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.623821  6975 raft_consensus.cc:1275] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.625691  6975 raft_consensus.cc:1275] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.677512  7705 consensus_queue.cc:1048] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.677676  7707 consensus_queue.cc:1048] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:55.693109  7522 consensus_queue.cc:1048] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:27:55.706763  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 53394us
W20260709 12:27:55.714596  6865 consensus_peers.cc:597] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.717238  6902 raft_consensus.cc:1275] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:27:55.738408  6866 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 116982us
W20260709 12:27:55.741238  6865 consensus_peers.cc:597] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.746677  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 70952.8us
I20260709 12:27:55.751878  7043 raft_consensus.cc:1275] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 99db41e57175426b9ceb11625b4455d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.796482  7043 raft_consensus.cc:1275] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 99db41e57175426b9ceb11625b4455d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.797817  7522 consensus_queue.cc:1048] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:27:55.801342  6865 consensus_peers.cc:597] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:55.810729  7683 consensus_queue.cc:1048] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:27:55.814441  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 44808.2us
I20260709 12:27:55.824970  6972 raft_consensus.cc:1275] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 99db41e57175426b9ceb11625b4455d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:55.834560  7575 consensus_queue.cc:1048] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:27:55.841136  6865 consensus_peers.cc:597] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:55.838270  7013 consensus_peers.cc:597] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:55.835376  7725 consensus_queue.cc:1048] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:27:55.849102  7683 consensus_queue.cc:1048] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:27:55.904790  7599 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:55.907742  7599 tablet_bootstrap.cc:492] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:55.908442  7599 ts_tablet_manager.cc:1403] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.609s	user 0.052s	sys 0.040s
I20260709 12:27:55.962333  7599 raft_consensus.cc:359] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:55.963064  7599 raft_consensus.cc:740] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:55.971962  7599 consensus_queue.cc:260] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:27:55.984050  6942 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 117541us
I20260709 12:27:55.985472  7599 ts_tablet_manager.cc:1434] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.001s
I20260709 12:27:56.010934  7599 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
W20260709 12:27:56.021364  7013 consensus_peers.cc:597] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:56.022128  7013 consensus_peers.cc:597] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:56.022168  6939 consensus_peers.cc:597] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:56.055676  6939 consensus_peers.cc:597] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:56.060735  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 56673.3us
W20260709 12:27:56.069154  6866 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 58634.7us
W20260709 12:27:56.109179  6865 consensus_peers.cc:597] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:56.109961  6865 consensus_peers.cc:597] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:56.093740  6724 catalog_manager.cc:5697] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:56.141208  6724 catalog_manager.cc:5697] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 reported cstate change: term changed from 1 to 2, leader changed from 38b3b964f82644c7b654e8042953a74e (127.29.37.65) to 99db41e57175426b9ceb11625b4455d9 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "99db41e57175426b9ceb11625b4455d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:56.145756  6865 consensus_peers.cc:597] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:56.216778  7599 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.217877  7599 tablet_bootstrap.cc:492] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.218693  7599 ts_tablet_manager.cc:1403] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.224s	user 0.024s	sys 0.017s
I20260709 12:27:56.222205  7599 raft_consensus.cc:359] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.230734  7599 raft_consensus.cc:740] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.231427  7599 consensus_queue.cc:260] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:27:56.238446  6939 consensus_peers.cc:597] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:56.239490  7599 ts_tablet_manager.cc:1434] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:27:56.240736  7599 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
W20260709 12:27:56.250586  6865 consensus_peers.cc:597] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:56.251636  6865 consensus_peers.cc:597] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:56.260617  6974 raft_consensus.cc:1275] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 99db41e57175426b9ceb11625b4455d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:56.272559  7599 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.273617  7599 tablet_bootstrap.cc:492] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.274432  7599 ts_tablet_manager.cc:1403] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.003s
I20260709 12:27:56.277488  7683 consensus_queue.cc:1048] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:56.277926  7599 raft_consensus.cc:359] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.278847  7599 raft_consensus.cc:740] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.279541  7599 consensus_queue.cc:260] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.281630  7599 ts_tablet_manager.cc:1434] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:27:56.282533  7599 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
W20260709 12:27:56.282426  6865 consensus_peers.cc:597] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:56.315747  7599 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.316694  7599 tablet_bootstrap.cc:492] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.317312  7599 ts_tablet_manager.cc:1403] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.008s
I20260709 12:27:56.319892  7599 raft_consensus.cc:359] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.320633  7599 raft_consensus.cc:740] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.321389  7599 consensus_queue.cc:260] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.323504  7599 ts_tablet_manager.cc:1434] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:27:56.324383  7599 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.345449  7599 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.346465  7599 tablet_bootstrap.cc:492] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.347203  7599 ts_tablet_manager.cc:1403] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.008s
I20260709 12:27:56.349602  7599 raft_consensus.cc:359] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.350270  7599 raft_consensus.cc:740] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.350889  7599 consensus_queue.cc:260] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.352996  7599 ts_tablet_manager.cc:1434] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:27:56.354404  7599 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.387104  7599 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.388268  7599 tablet_bootstrap.cc:492] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.389170  7599 ts_tablet_manager.cc:1403] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.015s
I20260709 12:27:56.392410  7599 raft_consensus.cc:359] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:56.393339  7599 raft_consensus.cc:740] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.394062  7599 consensus_queue.cc:260] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:56.396647  7599 ts_tablet_manager.cc:1434] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:27:56.399048  7599 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
W20260709 12:27:56.410115  6939 consensus_peers.cc:597] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:56.416700  7599 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.417606  7599 tablet_bootstrap.cc:492] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.418128  7599 ts_tablet_manager.cc:1403] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 12:27:56.420542  7599 raft_consensus.cc:359] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.421186  7599 raft_consensus.cc:740] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.421798  7599 consensus_queue.cc:260] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.424672  7599 ts_tablet_manager.cc:1434] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:27:56.425514  7599 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.441448  7599 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.442409  7599 tablet_bootstrap.cc:492] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.443076  7599 ts_tablet_manager.cc:1403] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:27:56.446420  7599 raft_consensus.cc:359] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.447109  7599 raft_consensus.cc:740] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.447947  7599 consensus_queue.cc:260] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.450256  7599 ts_tablet_manager.cc:1434] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:27:56.451570  7599 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.469478  7599 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.470428  7599 tablet_bootstrap.cc:492] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.471086  7599 ts_tablet_manager.cc:1403] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.002s
I20260709 12:27:56.473594  7599 raft_consensus.cc:359] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.474280  7599 raft_consensus.cc:740] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.474928  7599 consensus_queue.cc:260] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.476783  7599 ts_tablet_manager.cc:1434] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:56.477582  7599 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.490053  7599 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.490887  7599 tablet_bootstrap.cc:492] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.491457  7599 ts_tablet_manager.cc:1403] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.003s
I20260709 12:27:56.493244  7599 raft_consensus.cc:359] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.493727  7599 raft_consensus.cc:740] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.494199  7599 consensus_queue.cc:260] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.495622  7599 ts_tablet_manager.cc:1434] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:27:56.496618  7599 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.510176  7599 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.510928  7599 tablet_bootstrap.cc:492] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.511428  7599 ts_tablet_manager.cc:1403] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:27:56.513327  7599 raft_consensus.cc:359] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.513808  7599 raft_consensus.cc:740] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.514379  7599 consensus_queue.cc:260] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.516441  7599 ts_tablet_manager.cc:1434] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:56.520287  7599 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.534101  7599 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.534893  7599 tablet_bootstrap.cc:492] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.535435  7599 ts_tablet_manager.cc:1403] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.003s
I20260709 12:27:56.537660  7599 raft_consensus.cc:359] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.538384  7599 raft_consensus.cc:740] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.538970  7599 consensus_queue.cc:260] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.540967  7599 ts_tablet_manager.cc:1434] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:56.542093  7599 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.557415  7640 raft_consensus.cc:3060] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:56.557639  7599 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.558682  7599 tablet_bootstrap.cc:492] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.559350  7599 ts_tablet_manager.cc:1403] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.006s
I20260709 12:27:56.562510  7599 raft_consensus.cc:359] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.563169  7599 raft_consensus.cc:740] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.563828  7599 consensus_queue.cc:260] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.565984  7599 ts_tablet_manager.cc:1434] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.001s
I20260709 12:27:56.566849  7599 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.568346  7645 raft_consensus.cc:3060] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:56.591010  7599 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.591921  7599 tablet_bootstrap.cc:492] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.592592  7599 ts_tablet_manager.cc:1403] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.010s
I20260709 12:27:56.594877  7599 raft_consensus.cc:359] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.595570  7599 raft_consensus.cc:740] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.596278  7599 consensus_queue.cc:260] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.606593  7599 ts_tablet_manager.cc:1434] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:27:56.607342  7599 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.619627  7599 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.620826  7599 tablet_bootstrap.cc:492] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.621475  7599 ts_tablet_manager.cc:1403] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:27:56.623871  7599 raft_consensus.cc:359] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.624549  7599 raft_consensus.cc:740] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.625152  7599 consensus_queue.cc:260] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.627463  7599 ts_tablet_manager.cc:1434] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:27:56.628201  7599 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.629980  7640 raft_consensus.cc:3060] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:56.648419  7599 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.649338  7599 tablet_bootstrap.cc:492] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.649982  7599 ts_tablet_manager.cc:1403] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.003s
I20260709 12:27:56.652544  7599 raft_consensus.cc:359] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.653206  7599 raft_consensus.cc:740] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.653944  7599 consensus_queue.cc:260] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.656172  7599 ts_tablet_manager.cc:1434] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:27:56.657085  7599 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.669267  7599 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.670181  7599 tablet_bootstrap.cc:492] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.670892  7599 ts_tablet_manager.cc:1403] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.003s
I20260709 12:27:56.673094  7599 raft_consensus.cc:359] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.673681  7599 raft_consensus.cc:740] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.674206  7599 consensus_queue.cc:260] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.675859  7599 ts_tablet_manager.cc:1434] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:56.676911  7599 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.692901  7599 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.693796  7599 tablet_bootstrap.cc:492] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.694484  7599 ts_tablet_manager.cc:1403] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 12:27:56.696907  7599 raft_consensus.cc:359] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.697584  7599 raft_consensus.cc:740] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.698252  7599 consensus_queue.cc:260] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.700359  7599 ts_tablet_manager.cc:1434] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:27:56.702021  7599 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.718874  7599 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.719656  7599 tablet_bootstrap.cc:492] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.720218  7599 ts_tablet_manager.cc:1403] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:27:56.721976  7599 raft_consensus.cc:359] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:56.722499  7599 raft_consensus.cc:740] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.723114  7599 consensus_queue.cc:260] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:56.724668  7599 ts_tablet_manager.cc:1434] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:27:56.725422  7599 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.737040  7599 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.737839  7599 tablet_bootstrap.cc:492] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.738456  7599 ts_tablet_manager.cc:1403] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.004s
I20260709 12:27:56.740828  7599 raft_consensus.cc:359] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.741428  7599 raft_consensus.cc:740] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.742033  7599 consensus_queue.cc:260] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.744175  7599 ts_tablet_manager.cc:1434] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:56.744939  7599 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.757822  7599 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.758720  7599 tablet_bootstrap.cc:492] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.759333  7599 ts_tablet_manager.cc:1403] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.003s
I20260709 12:27:56.761623  7599 raft_consensus.cc:359] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.762507  7599 raft_consensus.cc:740] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.763155  7599 consensus_queue.cc:260] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:27:56.764707  7599 ts_tablet_manager.cc:1434] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:56.765350  7599 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.779012  7599 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.779712  7599 tablet_bootstrap.cc:492] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.780308  7599 ts_tablet_manager.cc:1403] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 12:27:56.782647  7599 raft_consensus.cc:359] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.783285  7599 raft_consensus.cc:740] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.783804  7599 consensus_queue.cc:260] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:27:56.785254  7599 ts_tablet_manager.cc:1434] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:56.786260  7599 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.798207  7599 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.799019  7599 tablet_bootstrap.cc:492] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.799623  7599 ts_tablet_manager.cc:1403] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:27:56.801661  7599 raft_consensus.cc:359] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:56.802373  7599 raft_consensus.cc:740] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.802958  7599 consensus_queue.cc:260] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:56.804976  7599 ts_tablet_manager.cc:1434] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:56.805915  7599 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Bootstrap starting.
I20260709 12:27:56.824738  7640 raft_consensus.cc:3060] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:56.826931  7599 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:56.827991  7599 tablet_bootstrap.cc:492] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Bootstrap complete.
I20260709 12:27:56.828601  7599 ts_tablet_manager.cc:1403] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 12:27:56.831105  7599 raft_consensus.cc:359] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:56.831743  7599 raft_consensus.cc:740] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Initialized, Role: FOLLOWER
I20260709 12:27:56.832384  7599 consensus_queue.cc:260] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:27:56.833837  7599 ts_tablet_manager.cc:1434] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:27:56.836650  7645 raft_consensus.cc:3060] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:56.878440  7640 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:57.048924  7645 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:57.050205  7640 raft_consensus.cc:3060] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:57.098259  7645 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:57.205832  7640 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:00.052462  7016 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.008s	sys 0.000s
W20260709 12:28:00.052858  7016 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.009s	sys 0.001s
W20260709 12:28:00.110021  6939 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.001s	sys 0.012s
W20260709 12:28:00.110455  6939 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.002s	sys 0.012s
W20260709 12:28:00.113229  6941 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.000s	sys 0.013s
W20260709 12:28:00.113580  6941 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.000s	sys 0.013s
W20260709 12:28:00.126322  7014 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.006s
W20260709 12:28:00.126669  7014 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.013s	sys 0.006s
I20260709 12:28:00.152168  7819 raft_consensus.cc:493] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99db41e57175426b9ceb11625b4455d9)
I20260709 12:28:00.152768  7819 raft_consensus.cc:515] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:28:00.158612  7819 leader_election.cc:290] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
W20260709 12:28:00.191152  7013 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 443312us
I20260709 12:28:00.233217  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99db41e57175426b9ceb11625b4455d9" is_pre_election: true
I20260709 12:28:00.236661  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b678720774394be3899ea14ff99628da" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c78bd8036f7040b99af7c501182cbe21" is_pre_election: true
I20260709 12:28:00.291198  7013 leader_election.cc:304] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 99db41e57175426b9ceb11625b4455d9, c78bd8036f7040b99af7c501182cbe21
I20260709 12:28:00.292088  7853 raft_consensus.cc:2749] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:28:00.903702  6756 debug-util.cc:398] Leaking SignalData structure 0x7b080041b660 after lost signal to thread 29848
W20260709 12:28:00.904912  6756 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6c00 after lost signal to thread 6759
W20260709 12:28:00.906124  6756 debug-util.cc:398] Leaking SignalData structure 0x7b080016f4c0 after lost signal to thread 6926
W20260709 12:28:00.907092  6756 debug-util.cc:398] Leaking SignalData structure 0x7b08001962e0 after lost signal to thread 7000
W20260709 12:28:00.908046  6756 debug-util.cc:398] Leaking SignalData structure 0x7b08001055e0 after lost signal to thread 7074
W20260709 12:28:00.909020  6756 debug-util.cc:398] Leaking SignalData structure 0x7b080021f880 after lost signal to thread 7671
I20260709 12:28:05.015304 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:05.104807  6897 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22a6861b97e443898d98ede72c787513"
dest_uuid: "99db41e57175426b9ceb11625b4455d9"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:37446
I20260709 12:28:05.105688  6897 raft_consensus.cc:606] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:05.108721  6897 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36d0e6da2d6d4fa6abb2757dff7de498"
dest_uuid: "99db41e57175426b9ceb11625b4455d9"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:37446
I20260709 12:28:05.109287  6897 raft_consensus.cc:606] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:05.123242  6975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef"
dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
mode: GRACEFUL
new_leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
 from {username='slave'} at 127.0.0.1:58364
I20260709 12:28:05.123836  6975 raft_consensus.cc:606] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Received request to transfer leadership to TS 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:28:05.127238  6897 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "580547b3f100485eba38ef98536bbc93"
dest_uuid: "99db41e57175426b9ceb11625b4455d9"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:37446
I20260709 12:28:05.127728  6897 raft_consensus.cc:606] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:05.141862  7045 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0"
dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:33440
I20260709 12:28:05.142392  7045 raft_consensus.cc:606] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:05.145602  6900 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9"
dest_uuid: "99db41e57175426b9ceb11625b4455d9"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:37446
I20260709 12:28:05.146170  6900 raft_consensus.cc:606] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:05.147475 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:28:05.149266 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:05.151098 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:05.295933  7825 raft_consensus.cc:993] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.320304  7876 raft_consensus.cc:1081] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
W20260709 12:28:05.345196  6865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:45171, user_credentials={real_user=slave}} blocked reactor thread for 54227.3us
I20260709 12:28:05.361668  7878 raft_consensus.cc:993] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.364125  7878 raft_consensus.cc:1081] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.366874  7859 raft_consensus.cc:993] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.371413  7874 raft_consensus.cc:1081] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.379751  7645 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22a6861b97e443898d98ede72c787513"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50674
I20260709 12:28:05.380605  7645 raft_consensus.cc:493] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:05.379642  7644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50676
I20260709 12:28:05.381279  7644 raft_consensus.cc:493] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:05.381633  7644 raft_consensus.cc:3060] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.386360  7645 raft_consensus.cc:3060] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.407588  7637 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "580547b3f100485eba38ef98536bbc93"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50674
I20260709 12:28:05.408466  7637 raft_consensus.cc:493] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:05.408988  7637 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:05.435168  7645 raft_consensus.cc:515] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.428959  7644 raft_consensus.cc:515] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:05.451164  7637 raft_consensus.cc:515] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.459697  7866 raft_consensus.cc:993] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21: : Instructing follower 3fb7d0187b544dfdb8487ecb2ff24a76 to start an election
I20260709 12:28:05.460650  7865 raft_consensus.cc:1081] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Signalling peer 3fb7d0187b544dfdb8487ecb2ff24a76 to start an election
I20260709 12:28:05.463008  7043 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef"
dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
 from {username='slave'} at 127.0.0.1:49104
I20260709 12:28:05.463680  7043 raft_consensus.cc:493] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:05.464071  7043 raft_consensus.cc:3060] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.466557  7644 leader_election.cc:290] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:28:05.477617  7645 leader_election.cc:290] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:05.485208  7043 raft_consensus.cc:515] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:28:05.485760  7637 leader_election.cc:290] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 3 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:05.489261  7879 raft_consensus.cc:993] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.496173  7863 raft_consensus.cc:1081] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.502805  7043 leader_election.cc:290] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:05.503506  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:05.504386  6897 raft_consensus.cc:3060] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.511026  7644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50674
I20260709 12:28:05.511646  7644 raft_consensus.cc:493] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:05.511340  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d22d1742e5946998079e448d9ab49ef" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:05.512053  7644 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:05.512256  6975 raft_consensus.cc:3055] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:05.512662  6975 raft_consensus.cc:740] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:28:05.513603  6975 consensus_queue.cc:260] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:28:05.515129  6975 raft_consensus.cc:3060] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.523517  7825 raft_consensus.cc:993] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.524088  7825 raft_consensus.cc:1081] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:05.525661  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:28:05.526059  7637 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36d0e6da2d6d4fa6abb2757dff7de498"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50674
I20260709 12:28:05.526419  7043 raft_consensus.cc:3055] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:05.526808  7043 raft_consensus.cc:740] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:28:05.526844  7637 raft_consensus.cc:493] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:05.527453  7637 raft_consensus.cc:3060] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.533231  7644 raft_consensus.cc:515] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.527889  7043 consensus_queue.cc:260] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:05.535862  7043 raft_consensus.cc:3060] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.538326  7644 leader_election.cc:290] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 3 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:05.546931  7637 raft_consensus.cc:515] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:05.549505  7637 leader_election.cc:290] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:28:05.550422  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d0e6da2d6d4fa6abb2757dff7de498" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:28:05.551060  7050 raft_consensus.cc:3060] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.552135  6897 raft_consensus.cc:2468] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:28:05.553562  7016 leader_election.cc:304] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:28:05.554490  7859 raft_consensus.cc:2804] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:05.555075  7859 raft_consensus.cc:697] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:28:05.556461  7859 consensus_queue.cc:237] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:28:05.560508  6975 raft_consensus.cc:2468] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:28:05.561866  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:05.562417  6973 raft_consensus.cc:3060] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.565748  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:05.566368  6974 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:05.571925  6971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:05.572604  6971 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:05.582631  7050 raft_consensus.cc:2468] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:05.589437  6971 raft_consensus.cc:2468] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 3.
I20260709 12:28:05.590417  7594 leader_election.cc:304] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:28:05.611755  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab076c84814416690df2fe770a98aa0" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:05.612448  6899 raft_consensus.cc:3060] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.612528  6900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a6861b97e443898d98ede72c787513" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:05.612732  6898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e12aacff6f754dfaa669e9651cce7dd9" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:05.613472  6898 raft_consensus.cc:3055] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:05.613857  6898 raft_consensus.cc:740] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:28:05.613152  6900 raft_consensus.cc:3055] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:05.614357  6900 raft_consensus.cc:740] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:28:05.614821  6898 consensus_queue.cc:260] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.616138  6898 raft_consensus.cc:3060] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:05.587568  7043 raft_consensus.cc:2468] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:05.634528  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36d0e6da2d6d4fa6abb2757dff7de498" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:05.629155  6721 catalog_manager.cc:5697] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 1 to 2, leader changed from c78bd8036f7040b99af7c501182cbe21 (127.29.37.67) to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 2 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:05.635195  6901 raft_consensus.cc:3055] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:05.635551  6901 raft_consensus.cc:740] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:28:05.636080  6899 raft_consensus.cc:2468] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:05.604362  6974 raft_consensus.cc:2468] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 3.
I20260709 12:28:05.613363  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "580547b3f100485eba38ef98536bbc93" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:05.637653  6902 raft_consensus.cc:3055] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:05.638005  6902 raft_consensus.cc:740] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:28:05.638948  6902 consensus_queue.cc:260] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.640347  6902 raft_consensus.cc:3060] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:05.643594  7593 leader_election.cc:304] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:28:05.599399  6973 raft_consensus.cc:2468] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:05.652652  7596 leader_election.cc:304] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
W20260709 12:28:05.689116  7594 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.68:41389, user_credentials={real_user=slave}} blocked reactor thread for 98929.9us
I20260709 12:28:05.700276  6900 consensus_queue.cc:260] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.702656  6901 consensus_queue.cc:260] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:05.703352  7887 raft_consensus.cc:2804] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:05.696610  7884 raft_consensus.cc:2804] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:05.710836  6900 raft_consensus.cc:3060] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.705821  6976 raft_consensus.cc:1240] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Rejecting Update request from peer 99db41e57175426b9ceb11625b4455d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:28:05.711205  6901 raft_consensus.cc:3060] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:05.706100  6902 raft_consensus.cc:2468] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 3.
I20260709 12:28:05.699419  6898 raft_consensus.cc:2468] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 3.
I20260709 12:28:05.724270  7596 leader_election.cc:304] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:28:05.721381  6901 raft_consensus.cc:2468] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:05.721909  7887 raft_consensus.cc:697] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 3 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:05.736828  7887 consensus_queue.cc:237] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.735733  7884 raft_consensus.cc:697] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:05.721365  7889 raft_consensus.cc:2804] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:05.754899  7889 raft_consensus.cc:697] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:05.742188  6900 raft_consensus.cc:2468] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:05.744110  7884 consensus_queue.cc:237] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
W20260709 12:28:05.773339  7593 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.67:41667, user_credentials={real_user=slave}} blocked reactor thread for 129974us
I20260709 12:28:05.770768  7889 consensus_queue.cc:237] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:05.760108  7887 raft_consensus.cc:2804] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:05.776981  7593 leader_election.cc:304] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:28:05.777552  7887 raft_consensus.cc:697] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 3 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
W20260709 12:28:05.780602  7596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:34951, user_credentials={real_user=slave}} blocked reactor thread for 56426.7us
I20260709 12:28:05.780491  7887 consensus_queue.cc:237] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.778342  7890 raft_consensus.cc:2804] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:05.783555  7890 raft_consensus.cc:697] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:05.784571  7890 consensus_queue.cc:237] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:05.857916  6723 catalog_manager.cc:5697] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from 99db41e57175426b9ceb11625b4455d9 (127.29.37.66) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:05.859838  6723 catalog_manager.cc:5697] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 2 to 3, leader changed from 99db41e57175426b9ceb11625b4455d9 (127.29.37.66) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 3 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:05.862059  6723 catalog_manager.cc:5697] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:05.918313  6723 catalog_manager.cc:5697] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 2 to 3, leader changed from 99db41e57175426b9ceb11625b4455d9 (127.29.37.66) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 3 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:05.922327  6723 catalog_manager.cc:5697] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from 99db41e57175426b9ceb11625b4455d9 (127.29.37.66) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:06.017445  6900 raft_consensus.cc:1275] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:06.019057  7747 consensus_queue.cc:1048] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:06.057514  6976 raft_consensus.cc:1275] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:06.058912  7873 consensus_queue.cc:1048] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:06.148870  6976 raft_consensus.cc:1275] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:06.150318  7889 consensus_queue.cc:1048] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:06.176180  6900 raft_consensus.cc:1275] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:06.182032  7887 consensus_queue.cc:1048] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:06.223109  6900 raft_consensus.cc:1275] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:06.227488  7887 consensus_queue.cc:1048] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:06.238795  6971 raft_consensus.cc:1275] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:06.244490  7884 consensus_queue.cc:1048] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:06.250466  7046 raft_consensus.cc:1275] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:06.252990  7889 consensus_queue.cc:1048] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:06.257510  6902 raft_consensus.cc:1275] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:06.263756  7884 consensus_queue.cc:1048] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:06.278968  7046 raft_consensus.cc:1275] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:06.279898  6902 raft_consensus.cc:1275] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:06.281152  7888 consensus_queue.cc:1048] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:06.281378  7889 consensus_queue.cc:1048] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:06.331597  6971 raft_consensus.cc:1275] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:06.338375  7889 consensus_queue.cc:1048] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:06.348029  6902 raft_consensus.cc:1275] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:06.349298  7889 consensus_queue.cc:1048] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:06.598003 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:06.646555  6976 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4"
dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:58364
I20260709 12:28:06.647107  6976 raft_consensus.cc:606] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:06.649710  6901 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f"
dest_uuid: "99db41e57175426b9ceb11625b4455d9"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:37446
I20260709 12:28:06.650310  6901 raft_consensus.cc:606] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:06.653231  7046 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77"
dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:33440
I20260709 12:28:06.653668  7046 raft_consensus.cc:606] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:06.656018  6976 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b"
dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
mode: GRACEFUL
new_leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
 from {username='slave'} at 127.0.0.1:58364
I20260709 12:28:06.656519  6976 raft_consensus.cc:606] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Received request to transfer leadership to TS 3fb7d0187b544dfdb8487ecb2ff24a76
I20260709 12:28:06.660554  6976 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb45dbc587b34cb7843e48180e0219c3"
dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:58364
I20260709 12:28:06.661060  6976 raft_consensus.cc:606] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:06.661047  7859 raft_consensus.cc:993] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:06.661827  7747 raft_consensus.cc:1081] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:06.663385  7637 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50676
I20260709 12:28:06.663975  7637 raft_consensus.cc:493] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:06.664350  7637 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:06.662235 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:06.667764 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:06.669169 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:06.673913  7637 raft_consensus.cc:515] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:06.678956  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:06.680495  7046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9875aff5560f4adc82ed2881b0719c77" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:28:06.681113  7046 raft_consensus.cc:3055] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:06.681440  7046 raft_consensus.cc:740] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:28:06.682288  7046 consensus_queue.cc:260] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:06.683401  7046 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:06.683524  6976 raft_consensus.cc:3060] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:06.689888  6976 raft_consensus.cc:2468] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 3.
I20260709 12:28:06.692051  7593 leader_election.cc:304] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:28:06.692951  7890 raft_consensus.cc:2804] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:06.693434  7890 raft_consensus.cc:697] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 3 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:06.694415  7637 leader_election.cc:290] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 3 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:28:06.694370  7890 consensus_queue.cc:237] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:06.713338  7046 raft_consensus.cc:2468] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 3.
I20260709 12:28:06.716666  6723 catalog_manager.cc:5697] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 2 to 3, leader changed from 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 3 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:06.764635  7834 raft_consensus.cc:993] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:06.765122  7866 raft_consensus.cc:1081] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:06.766600  7637 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50668
I20260709 12:28:06.767133  7637 raft_consensus.cc:493] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:06.767508  7637 raft_consensus.cc:3060] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.775436  7637 raft_consensus.cc:515] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:06.777765  7637 leader_election.cc:290] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:06.777873  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:06.778484  6901 raft_consensus.cc:3060] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.779062  6971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02d86f9e2d5d4ca195483d90646330a4" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:06.779748  6971 raft_consensus.cc:3055] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:06.780071  6971 raft_consensus.cc:740] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:28:06.780854  6971 consensus_queue.cc:260] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:06.782181  6971 raft_consensus.cc:3060] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.784255  7866 raft_consensus.cc:993] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21: : Instructing follower 3fb7d0187b544dfdb8487ecb2ff24a76 to start an election
I20260709 12:28:06.784768  7834 raft_consensus.cc:1081] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Signalling peer 3fb7d0187b544dfdb8487ecb2ff24a76 to start an election
I20260709 12:28:06.786195  6901 raft_consensus.cc:2468] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:06.787313  7596 leader_election.cc:304] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:28:06.787935  7890 raft_consensus.cc:2804] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:06.788429  7890 raft_consensus.cc:697] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:06.789304  7890 consensus_queue.cc:237] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:06.790100  6971 raft_consensus.cc:2468] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:06.791044  7046 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b"
dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
 from {username='slave'} at 127.0.0.1:49104
I20260709 12:28:06.791620  7046 raft_consensus.cc:493] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:06.792001  7046 raft_consensus.cc:3060] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.797856  7046 raft_consensus.cc:515] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:28:06.798575  6723 catalog_manager.cc:5697] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from c78bd8036f7040b99af7c501182cbe21 (127.29.37.67) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:06.799602  7046 leader_election.cc:290] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:06.801460  6971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:06.802110  6971 raft_consensus.cc:3055] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:06.802487  6971 raft_consensus.cc:740] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:28:06.800272  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85b20398cd74f5888b96dc53ab9179b" candidate_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:06.803193  6901 raft_consensus.cc:3060] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.803373  6971 consensus_queue.cc:260] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:28:06.804826  6971 raft_consensus.cc:3060] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.808698  6901 raft_consensus.cc:2468] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:28:06.809731  7016 leader_election.cc:304] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:28:06.810544  7859 raft_consensus.cc:2804] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:06.811019  7859 raft_consensus.cc:697] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Leader. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:28:06.811616  6971 raft_consensus.cc:2468] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb7d0187b544dfdb8487ecb2ff24a76 in term 2.
I20260709 12:28:06.811967  7859 consensus_queue.cc:237] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } }
I20260709 12:28:06.821681  6723 catalog_manager.cc:5697] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 reported cstate change: term changed from 1 to 2, leader changed from c78bd8036f7040b99af7c501182cbe21 (127.29.37.67) to 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68). New cstate: current_term: 2 leader_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:06.934935  7825 raft_consensus.cc:993] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:06.935381  7863 raft_consensus.cc:1081] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:06.936666  7641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50674
I20260709 12:28:06.937202  7641 raft_consensus.cc:493] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:06.937460  7641 raft_consensus.cc:3060] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.943449  7641 raft_consensus.cc:515] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:06.945868  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:06.946339  7641 leader_election.cc:290] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:06.946432  6901 raft_consensus.cc:3055] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:06.946955  6901 raft_consensus.cc:740] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99db41e57175426b9ceb11625b4455d9, State: Running, Role: LEADER
I20260709 12:28:06.946987  6971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5257ef3bead24ca48d68d7b8415e116f" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:06.947664  6971 raft_consensus.cc:3060] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.947692  6901 consensus_queue.cc:260] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:06.949375  6901 raft_consensus.cc:3060] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.954742  6971 raft_consensus.cc:2468] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:06.955471  6901 raft_consensus.cc:2468] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:06.956552  7596 leader_election.cc:304] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:28:06.957300  7890 raft_consensus.cc:2804] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:06.957762  7890 raft_consensus.cc:697] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:06.958570  7890 consensus_queue.cc:237] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:06.968362  6721 catalog_manager.cc:5697] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from 99db41e57175426b9ceb11625b4455d9 (127.29.37.66) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:07.117687  6971 raft_consensus.cc:1275] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:07.119127  7890 consensus_queue.cc:1048] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:07.140456  7043 raft_consensus.cc:1275] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 3 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:07.146452  7889 consensus_queue.cc:1048] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:07.203187  7834 raft_consensus.cc:993] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:07.203712  7866 raft_consensus.cc:1081] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:07.205690  7637 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb45dbc587b34cb7843e48180e0219c3"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50668
I20260709 12:28:07.206154  7637 raft_consensus.cc:493] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:07.206521  7637 raft_consensus.cc:3060] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.212612  7637 raft_consensus.cc:515] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:07.214915  7637 leader_election.cc:290] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:28:07.215175  6971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb45dbc587b34cb7843e48180e0219c3" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:07.215423  7046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb45dbc587b34cb7843e48180e0219c3" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:28:07.215801  6971 raft_consensus.cc:3055] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:07.216090  7046 raft_consensus.cc:3060] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.216841  6971 raft_consensus.cc:740] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:28:07.217564  6971 consensus_queue.cc:260] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:07.218725  6971 raft_consensus.cc:3060] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.222070  7046 raft_consensus.cc:2468] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:07.223351  7594 leader_election.cc:304] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:28:07.224215  7890 raft_consensus.cc:2804] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:07.224725  7890 raft_consensus.cc:697] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:07.225179  6971 raft_consensus.cc:2468] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:07.225550  7890 consensus_queue.cc:237] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:07.234845  6721 catalog_manager.cc:5697] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from c78bd8036f7040b99af7c501182cbe21 (127.29.37.67) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.250962  6971 raft_consensus.cc:1275] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.252357  7889 consensus_queue.cc:1048] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.262662  6902 raft_consensus.cc:1275] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.264111  7889 consensus_queue.cc:1048] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.347262  6976 raft_consensus.cc:1275] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.348377  7859 consensus_queue.cc:1048] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.357177  6902 raft_consensus.cc:1275] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 3fb7d0187b544dfdb8487ecb2ff24a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.359018  7859 consensus_queue.cc:1048] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.397533  6902 raft_consensus.cc:1275] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.398677  7890 consensus_queue.cc:1048] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.418200  6976 raft_consensus.cc:1275] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.419517  7892 consensus_queue.cc:1048] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.725584 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:07.763684  6971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866"
dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:58364
I20260709 12:28:07.764228  6971 raft_consensus.cc:606] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:07.767449  6971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc"
dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:58364
I20260709 12:28:07.768044  6971 raft_consensus.cc:606] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:07.768677  7046 raft_consensus.cc:1275] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.770716  7043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01"
dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:33440
I20260709 12:28:07.770659  7892 consensus_queue.cc:1048] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.771330  7043 raft_consensus.cc:606] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:07.774475  6971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e962d48b16794d78ad1a367540885945"
dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:58364
I20260709 12:28:07.775120  6971 raft_consensus.cc:606] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:07.781311  6971 raft_consensus.cc:1275] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.783504  7892 consensus_queue.cc:1048] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.787972  6971 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10"
dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
mode: GRACEFUL
new_leader_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:58364
I20260709 12:28:07.788663  6971 raft_consensus.cc:606] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Received request to transfer leadership to TS 38b3b964f82644c7b654e8042953a74e
I20260709 12:28:07.793746 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:07.795477 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:07.796239 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:07.864163  7866 raft_consensus.cc:993] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:07.864694  7834 raft_consensus.cc:1081] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:07.865856  7641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50668
I20260709 12:28:07.866292  7641 raft_consensus.cc:493] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:07.866539  7641 raft_consensus.cc:3060] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.871474  7641 raft_consensus.cc:515] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:07.873672  6902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:07.873471  7641 leader_election.cc:290] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:07.873956  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46b2ca102c7b4575a89399cac44ee866" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:07.874290  6902 raft_consensus.cc:3060] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.874609  6976 raft_consensus.cc:3055] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:07.874920  6976 raft_consensus.cc:740] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:28:07.875707  6976 consensus_queue.cc:260] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:07.876960  6976 raft_consensus.cc:3060] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.880592  6902 raft_consensus.cc:2468] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:07.881789  7596 leader_election.cc:304] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:28:07.882652  7892 raft_consensus.cc:2804] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:07.883111  7892 raft_consensus.cc:697] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:07.883471  6976 raft_consensus.cc:2468] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:07.884056  7892 consensus_queue.cc:237] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:07.896030  6721 catalog_manager.cc:5697] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from c78bd8036f7040b99af7c501182cbe21 (127.29.37.67) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:08.078495  7866 raft_consensus.cc:993] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:08.079033  7866 raft_consensus.cc:1081] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:08.080271  7834 raft_consensus.cc:993] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:08.080734  7866 raft_consensus.cc:1081] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:08.080657  7645 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50668
I20260709 12:28:08.082382  7645 raft_consensus.cc:493] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:08.082933  7645 raft_consensus.cc:3060] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.083289  7641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e962d48b16794d78ad1a367540885945"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50668
I20260709 12:28:08.084270  7641 raft_consensus.cc:493] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:08.084619  7641 raft_consensus.cc:3060] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.088977  7866 raft_consensus.cc:993] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:08.090286  7866 raft_consensus.cc:1081] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:08.093412  7637 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50668
I20260709 12:28:08.093977  7645 raft_consensus.cc:515] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:08.093995  7637 raft_consensus.cc:493] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:08.095175  7637 raft_consensus.cc:3060] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.096560  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:08.097118  6901 raft_consensus.cc:3060] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.097561  7641 raft_consensus.cc:515] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:08.099208  6971 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b68394604b04367a2e1169ce7623abc" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:08.099884  6971 raft_consensus.cc:3055] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:08.100178  6971 raft_consensus.cc:740] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:28:08.095963  7645 leader_election.cc:290] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951), c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667)
I20260709 12:28:08.101025  7043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e962d48b16794d78ad1a367540885945" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:28:08.101614  7043 raft_consensus.cc:3060] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.101801  6976 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e962d48b16794d78ad1a367540885945" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:08.100977  6971 consensus_queue.cc:260] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:08.102734  6976 raft_consensus.cc:3055] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:08.103098  6976 raft_consensus.cc:740] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:28:08.103480  6971 raft_consensus.cc:3060] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.103833  6976 consensus_queue.cc:260] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:08.104894  6976 raft_consensus.cc:3060] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.105670  7637 raft_consensus.cc:515] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:08.107484  7637 leader_election.cc:290] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:28:08.108382  7046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:28:08.108950  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efdd38fdcb5f4e2198f1180e3fe51a10" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c78bd8036f7040b99af7c501182cbe21"
I20260709 12:28:08.108999  7046 raft_consensus.cc:3060] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.109539  6973 raft_consensus.cc:3055] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:08.109854  6973 raft_consensus.cc:740] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c78bd8036f7040b99af7c501182cbe21, State: Running, Role: LEADER
I20260709 12:28:08.100500  7641 leader_election.cc:290] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Requested vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389)
I20260709 12:28:08.110584  6973 consensus_queue.cc:260] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:08.112821  7043 raft_consensus.cc:2468] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:08.117295  6971 raft_consensus.cc:2468] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:08.113400  6901 raft_consensus.cc:2468] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:08.117774  6973 raft_consensus.cc:3060] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:08.116509  6976 raft_consensus.cc:2468] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:08.118373  7594 leader_election.cc:304] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 
I20260709 12:28:08.118757  7593 leader_election.cc:304] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:28:08.119234  7892 raft_consensus.cc:2804] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:08.119870  7890 raft_consensus.cc:2804] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:08.120366  7890 raft_consensus.cc:697] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:08.121233  7892 raft_consensus.cc:697] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:08.122359  7892 consensus_queue.cc:237] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:08.124634  6973 raft_consensus.cc:2468] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:08.121263  7890 consensus_queue.cc:237] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } }
I20260709 12:28:08.124639  7046 raft_consensus.cc:2468] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 2.
I20260709 12:28:08.126138  7593 leader_election.cc:304] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21; no voters: 
I20260709 12:28:08.128147  7889 raft_consensus.cc:2804] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:08.130849  7889 raft_consensus.cc:697] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:08.135121  7889 consensus_queue.cc:237] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:08.136158  7934 raft_consensus.cc:993] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76: : Instructing follower 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:08.136734  7859 raft_consensus.cc:1081] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Signalling peer 38b3b964f82644c7b654e8042953a74e to start an election
I20260709 12:28:08.139241  7641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01"
dest_uuid: "38b3b964f82644c7b654e8042953a74e"
 from {username='slave'} at 127.0.0.1:50676
I20260709 12:28:08.139878  7641 raft_consensus.cc:493] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:08.140221  7641 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:08.146343  7641 raft_consensus.cc:515] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:08.146903  6721 catalog_manager.cc:5697] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from c78bd8036f7040b99af7c501182cbe21 (127.29.37.67) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:08.148480  7046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76"
I20260709 12:28:08.149076  7046 raft_consensus.cc:3055] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:08.149392  7046 raft_consensus.cc:740] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3fb7d0187b544dfdb8487ecb2ff24a76, State: Running, Role: LEADER
I20260709 12:28:08.149977  6901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afde26a33c5c45cfb6b2087601e4ce01" candidate_uuid: "38b3b964f82644c7b654e8042953a74e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "99db41e57175426b9ceb11625b4455d9"
I20260709 12:28:08.150588  6901 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:08.150291  7046 consensus_queue.cc:260] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:08.152500  7641 leader_election.cc:290] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 3 election: Requested vote from peers 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68:41389), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:28:08.151639  7046 raft_consensus.cc:3060] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:08.157328  6901 raft_consensus.cc:2468] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 3.
I20260709 12:28:08.158725  7596 leader_election.cc:304] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9; no voters: 
I20260709 12:28:08.159732  7889 raft_consensus.cc:2804] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:08.160382  7889 raft_consensus.cc:697] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 3 LEADER]: Becoming Leader. State: Replica: 38b3b964f82644c7b654e8042953a74e, State: Running, Role: LEADER
I20260709 12:28:08.161226  7889 consensus_queue.cc:237] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:08.162588  7046 raft_consensus.cc:2468] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38b3b964f82644c7b654e8042953a74e in term 3.
I20260709 12:28:08.166023  6721 catalog_manager.cc:5697] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from c78bd8036f7040b99af7c501182cbe21 (127.29.37.67) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:08.167361  6721 catalog_manager.cc:5697] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 1 to 2, leader changed from c78bd8036f7040b99af7c501182cbe21 (127.29.37.67) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 2 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:08.187721  6721 catalog_manager.cc:5697] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e reported cstate change: term changed from 2 to 3, leader changed from 3fb7d0187b544dfdb8487ecb2ff24a76 (127.29.37.68) to 38b3b964f82644c7b654e8042953a74e (127.29.37.65). New cstate: current_term: 3 leader_uuid: "38b3b964f82644c7b654e8042953a74e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:08.293246  6901 raft_consensus.cc:1275] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.294502  7889 consensus_queue.cc:1048] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:08.296306  6973 raft_consensus.cc:1275] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.297501  7889 consensus_queue.cc:1048] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:08.571954  6902 raft_consensus.cc:1275] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:08.573158  7890 consensus_queue.cc:1048] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:08.591369  7046 raft_consensus.cc:1275] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 3 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:08.595547  7889 consensus_queue.cc:1048] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:08.657605  7046 raft_consensus.cc:1275] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.658927  7889 consensus_queue.cc:1048] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:08.660641  6971 raft_consensus.cc:1275] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.662364  7889 consensus_queue.cc:1048] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:28:08.666297  7043 raft_consensus.cc:1275] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.670069  7889 consensus_queue.cc:1048] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:08.676611  6973 raft_consensus.cc:1275] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.679207  7945 consensus_queue.cc:1048] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:08.680032  6901 raft_consensus.cc:1275] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.685380  7889 consensus_queue.cc:1048] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:08.690316  6973 raft_consensus.cc:1275] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Refusing update from remote peer 38b3b964f82644c7b654e8042953a74e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.692246  7892 consensus_queue.cc:1048] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:08.851852 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:08.887111 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:08.888550 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:08.889263 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:08.927211 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:08.960841 29845 tablet_server.cc:179] TabletServer@127.29.37.65:45171 shutting down...
W20260709 12:28:08.966502  7013 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 63 similar messages]
W20260709 12:28:08.970762  7013 consensus_peers.cc:597] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:08.992985  7013 consensus_peers.cc:597] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:08.993640  7013 consensus_peers.cc:597] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.013729  7013 consensus_peers.cc:597] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.024952  6865 consensus_peers.cc:597] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.044708 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:28:09.045269  7013 consensus_peers.cc:597] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.045914 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.047078 29845 raft_consensus.cc:2243] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.048416 29845 raft_consensus.cc:2272] T e962d48b16794d78ad1a367540885945 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.050974  6865 consensus_peers.cc:597] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.051656  6865 consensus_peers.cc:597] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.053567 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.054603 29845 raft_consensus.cc:2243] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.055119 29845 raft_consensus.cc:2272] T 122d63cf583043458b719df07a6a694d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.058140 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.059103 29845 raft_consensus.cc:2243] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.059604 29845 raft_consensus.cc:2272] T e11f6b3dd5fe466d984f796948818c0d P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.062543 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.063432 29845 raft_consensus.cc:2243] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.064358 29845 raft_consensus.cc:2272] T 9875aff5560f4adc82ed2881b0719c77 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.067312 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.068166 29845 raft_consensus.cc:2243] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.069306 29845 raft_consensus.cc:2272] T e12aacff6f754dfaa669e9651cce7dd9 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.073092 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.074023 29845 raft_consensus.cc:2243] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.074584 29845 raft_consensus.cc:2272] T 4ac46831fcf945d194fd7677ada23856 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.077440 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.078325 29845 raft_consensus.cc:2243] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.079593 29845 raft_consensus.cc:2272] T d2fcd7006bc840a0bb5ea27365793f86 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.080415  6865 consensus_peers.cc:597] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.084089  7013 consensus_peers.cc:597] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.084388 29845 tablet_replica.cc:333] stopping tablet replica
W20260709 12:28:09.084599  6939 consensus_peers.cc:597] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.085304 29845 raft_consensus.cc:2243] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.086184 29845 raft_consensus.cc:2272] T 580547b3f100485eba38ef98536bbc93 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.089670 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.090372 29845 raft_consensus.cc:2243] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.091553 29845 raft_consensus.cc:2272] T cd4c913d08ea42b09038a2a80ecfa3b6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.094599 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.095176 29845 raft_consensus.cc:2243] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.095732 29845 raft_consensus.cc:2272] T 0aa7e52e7c5b4039a78f136142693567 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.098217 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.098995 29845 raft_consensus.cc:2243] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.099581 29845 raft_consensus.cc:2272] T a690e13ef48d439cb389b897c6b9fcc6 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.102056  6865 consensus_peers.cc:597] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.102500 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.103835 29845 raft_consensus.cc:2243] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.104743 29845 raft_consensus.cc:2272] T 46b2ca102c7b4575a89399cac44ee866 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.107038  6865 consensus_peers.cc:597] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.107854 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.108769 29845 raft_consensus.cc:2243] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.109239 29845 raft_consensus.cc:2272] T d82cf9bc8bd54dc08a7e03270cacde99 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.111618 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.112442 29845 raft_consensus.cc:2243] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.113283 29845 raft_consensus.cc:2272] T 02d86f9e2d5d4ca195483d90646330a4 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.115300 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.115878 29845 raft_consensus.cc:2243] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.116276 29845 raft_consensus.cc:2272] T f3ed0915d1da4514bc4c51eddfcf5e31 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.118213 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.118794 29845 raft_consensus.cc:2243] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.119177 29845 raft_consensus.cc:2272] T 4e2b62ad0158424696dcdf73faa4f675 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.121079 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.121703 29845 raft_consensus.cc:2243] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.122115 29845 raft_consensus.cc:2272] T a38ba9e3e56242779781f03654f3dfd3 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.124073 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.124687 29845 raft_consensus.cc:2243] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:09.125074  6865 consensus_peers.cc:597] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.125097 29845 raft_consensus.cc:2272] T 7c55f4a15fa641de92c4922bc65c06be P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.127385 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.128058 29845 raft_consensus.cc:2243] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.128780 29845 raft_consensus.cc:2272] T 22a6861b97e443898d98ede72c787513 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.130934 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.131708 29845 raft_consensus.cc:2243] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.132673 29845 raft_consensus.cc:2272] T 36d0e6da2d6d4fa6abb2757dff7de498 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.135296 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.135829 29845 raft_consensus.cc:2243] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.136538 29845 raft_consensus.cc:2272] T 5257ef3bead24ca48d68d7b8415e116f P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.136564  7013 consensus_peers.cc:597] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.139272 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.140026 29845 raft_consensus.cc:2243] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.140494 29845 raft_consensus.cc:2272] T e381f83716b64b7982f5f0c9c63d348f P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.142436 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.143008 29845 raft_consensus.cc:2243] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.143406 29845 raft_consensus.cc:2272] T 86c78f0ec1af4414a7e52c69f19b6c27 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.145263 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.145781 29845 raft_consensus.cc:2243] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.146199 29845 raft_consensus.cc:2272] T 8203d585d06840daadbcfbb32fb2f822 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.148205 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.148717 29845 raft_consensus.cc:2243] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.149089 29845 raft_consensus.cc:2272] T 07ecb64e39814c7a961ebb058d596cbc P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.151317 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.152007 29845 raft_consensus.cc:2243] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.152432 29845 raft_consensus.cc:2272] T f932aaa29a8443d4b251dea7c9fc0f54 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.154863 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.155376 29845 raft_consensus.cc:2243] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.155797 29845 raft_consensus.cc:2272] T 04e9253071944b29a6116b8d9f64ef3a P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.157790 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.158366 29845 raft_consensus.cc:2243] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.159066 29845 raft_consensus.cc:2272] T cb45dbc587b34cb7843e48180e0219c3 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.161135 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.161823 29845 raft_consensus.cc:2243] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.162282 29845 raft_consensus.cc:2272] T 70fa5eb82f684b449d82176587520e4d P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.164096 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.164872 29845 raft_consensus.cc:2243] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.165547 29845 raft_consensus.cc:2272] T 8ab076c84814416690df2fe770a98aa0 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.167431 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.167927 29845 raft_consensus.cc:2243] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.168259 29845 raft_consensus.cc:2272] T 847f2ce68f10434cb350d63a6573b3bd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.170135 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.170608 29845 raft_consensus.cc:2243] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.171257 29845 raft_consensus.cc:2272] T afde26a33c5c45cfb6b2087601e4ce01 P 38b3b964f82644c7b654e8042953a74e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.173311 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.173797 29845 raft_consensus.cc:2243] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.174162 29845 raft_consensus.cc:2272] T 8dc898afa9a94274850432d379c5203e P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.176218 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.176823 29845 raft_consensus.cc:2243] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.177402 29845 raft_consensus.cc:2272] T c92afb4b5d234fd48fe0f24ec262964a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.179924 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.180364 29845 raft_consensus.cc:2243] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.181010 29845 raft_consensus.cc:2272] T 7b68394604b04367a2e1169ce7623abc P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.183326 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.183840 29845 raft_consensus.cc:2243] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.184280 29845 raft_consensus.cc:2272] T 93204a5a0d0742ad9cd21a8a280af392 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.186436 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.186940 29845 raft_consensus.cc:2243] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.187330 29845 raft_consensus.cc:2272] T 8c77ef2e07ff418b996e1c72415228d6 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.189376 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.189996 29845 raft_consensus.cc:2243] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.190476 29845 raft_consensus.cc:2272] T 0c2693326c6b49989473c3de8c70d40a P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.193022 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.193574 29845 raft_consensus.cc:2243] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.193996 29845 raft_consensus.cc:2272] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.196811 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.197356 29845 raft_consensus.cc:2243] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.197770 29845 raft_consensus.cc:2272] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.199934 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.200430 29845 raft_consensus.cc:2243] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.200824 29845 raft_consensus.cc:2272] T 718060ceb1df4264ac14b8e4c8f9c007 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.202795 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.203320 29845 raft_consensus.cc:2243] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.203800 29845 raft_consensus.cc:2272] T e7eee84ade114ccaa536076ae9b04ac2 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.205854 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.206444 29845 raft_consensus.cc:2243] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.206918 29845 raft_consensus.cc:2272] T 3b065e90800f40379b501a6238c6fdcd P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.208909 29845 tablet_replica.cc:333] stopping tablet replica
W20260709 12:28:09.208928  6939 consensus_peers.cc:597] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.209846 29845 raft_consensus.cc:2243] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.210769 29845 raft_consensus.cc:2272] T efdd38fdcb5f4e2198f1180e3fe51a10 P 38b3b964f82644c7b654e8042953a74e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.213246 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:09.213704 29845 raft_consensus.cc:2243] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.214069 29845 raft_consensus.cc:2272] T 251b19e52f0146aa9a9cfc985e2763b4 P 38b3b964f82644c7b654e8042953a74e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.228423  6865 consensus_peers.cc:597] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.251010  6865 consensus_peers.cc:597] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.274590  6939 consensus_peers.cc:597] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.282557 29845 tablet_server.cc:196] TabletServer@127.29.37.65:45171 shutdown complete.
W20260709 12:28:09.295164  6865 consensus_peers.cc:597] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.295742  7013 consensus_peers.cc:597] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.318077  6939 consensus_peers.cc:597] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.318970  6939 consensus_peers.cc:597] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.325922  6939 consensus_peers.cc:597] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.331502  7013 consensus_peers.cc:597] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.360615  7013 consensus_peers.cc:597] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.363746  6865 consensus_peers.cc:597] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.371658  6939 consensus_peers.cc:597] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.376066 29845 tablet_server.cc:179] TabletServer@127.29.37.66:0 shutting down...
W20260709 12:28:09.383337  6942 consensus_peers.cc:597] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.390972  7016 consensus_peers.cc:597] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.416402 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:09.417277 29845 tablet_replica.cc:333] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.418036 29845 raft_consensus.cc:2243] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.418905 29845 raft_consensus.cc:2272] T 3b065e90800f40379b501a6238c6fdcd P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.421058 29845 tablet_replica.cc:333] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.421674 29845 raft_consensus.cc:2243] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.422554 29845 raft_consensus.cc:2272] T f932aaa29a8443d4b251dea7c9fc0f54 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.424584 29845 tablet_replica.cc:333] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.425299 29845 raft_consensus.cc:2243] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.425751 29845 raft_consensus.cc:2272] T 5df4f6356b53461e89321a089ab9b65f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.428279 29845 tablet_replica.cc:333] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.428930 29845 raft_consensus.cc:2243] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.429445 29845 raft_consensus.cc:2272] T a690e13ef48d439cb389b897c6b9fcc6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.429939  7013 consensus_peers.cc:597] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.431911 29845 tablet_replica.cc:333] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.432879 29845 raft_consensus.cc:2243] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.433427 29845 raft_consensus.cc:2272] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.436188 29845 tablet_replica.cc:333] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.436964 29845 raft_consensus.cc:2243] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.437443 29845 raft_consensus.cc:2272] T 580547b3f100485eba38ef98536bbc93 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.437551  6942 consensus_peers.cc:597] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.440150  7013 consensus_peers.cc:597] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.440341 29845 tablet_replica.cc:333] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.441233 29845 raft_consensus.cc:2243] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.442535 29845 raft_consensus.cc:2272] T 718060ceb1df4264ac14b8e4c8f9c007 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.445362 29845 tablet_replica.cc:333] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.446177 29845 raft_consensus.cc:2243] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.446700 29845 raft_consensus.cc:2272] T 4d22d1742e5946998079e448d9ab49ef P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.449334 29845 tablet_replica.cc:333] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.450167 29845 raft_consensus.cc:2243] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.451174 29845 raft_consensus.cc:2272] T cd4c913d08ea42b09038a2a80ecfa3b6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.451449  6942 consensus_peers.cc:597] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.453941 29845 tablet_replica.cc:333] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.454823 29845 raft_consensus.cc:2243] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.455817 29845 raft_consensus.cc:2272] T e381f83716b64b7982f5f0c9c63d348f P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.458389 29845 tablet_replica.cc:333] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.459110 29845 raft_consensus.cc:2243] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.459534 29845 raft_consensus.cc:2272] T a2d766dbe21d42a0ab36d005d89289c9 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.459715  6939 consensus_peers.cc:597] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 -> Peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Couldn't send request to peer 38b3b964f82644c7b654e8042953a74e. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.461534 29845 tablet_replica.cc:333] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.462157 29845 raft_consensus.cc:2243] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.462604 29845 raft_consensus.cc:2272] T e11f6b3dd5fe466d984f796948818c0d P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.464563 29845 tablet_replica.cc:333] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.465050 29845 raft_consensus.cc:2243] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.465382 29845 raft_consensus.cc:2272] T 5257ef3bead24ca48d68d7b8415e116f P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.467232 29845 tablet_replica.cc:333] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.467706 29845 raft_consensus.cc:2243] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.468044 29845 raft_consensus.cc:2272] T 36d0e6da2d6d4fa6abb2757dff7de498 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.470265 29845 tablet_replica.cc:333] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.470889 29845 raft_consensus.cc:2243] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.471354 29845 raft_consensus.cc:2272] T bfbf51a7748b461686ff7db762bba509 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.473344 29845 tablet_replica.cc:333] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.473954 29845 raft_consensus.cc:2243] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.474404 29845 raft_consensus.cc:2272] T 02d86f9e2d5d4ca195483d90646330a4 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.476488 29845 tablet_replica.cc:333] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.477330 29845 raft_consensus.cc:2243] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.478446 29845 raft_consensus.cc:2272] T d82cf9bc8bd54dc08a7e03270cacde99 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.481069 29845 tablet_replica.cc:333] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.481822 29845 raft_consensus.cc:2243] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.482470 29845 raft_consensus.cc:2272] T 46b2ca102c7b4575a89399cac44ee866 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.484506 29845 tablet_replica.cc:333] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.485210 29845 raft_consensus.cc:2243] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.485687 29845 raft_consensus.cc:2272] T 8dc898afa9a94274850432d379c5203e P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.487859  6942 consensus_peers.cc:597] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.488158 29845 tablet_replica.cc:333] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.488762 29845 raft_consensus.cc:2243] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.489126 29845 raft_consensus.cc:2272] T afde26a33c5c45cfb6b2087601e4ce01 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.491911 29845 tablet_replica.cc:333] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.492453 29845 raft_consensus.cc:2243] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.492866 29845 raft_consensus.cc:2272] T 8ab076c84814416690df2fe770a98aa0 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.494657 29845 tablet_replica.cc:333] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.495157 29845 raft_consensus.cc:2243] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.495532 29845 raft_consensus.cc:2272] T 847f2ce68f10434cb350d63a6573b3bd P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.497613 29845 tablet_replica.cc:333] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.498127 29845 raft_consensus.cc:2243] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.498574 29845 raft_consensus.cc:2272] T a85b20398cd74f5888b96dc53ab9179b P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.500391 29845 tablet_replica.cc:333] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.500929 29845 raft_consensus.cc:2243] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.501323 29845 raft_consensus.cc:2272] T 22a6861b97e443898d98ede72c787513 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.503261 29845 tablet_replica.cc:333] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.503782 29845 raft_consensus.cc:2243] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.504526 29845 raft_consensus.cc:2272] T 7c55f4a15fa641de92c4922bc65c06be P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.506744 29845 tablet_replica.cc:333] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.507462 29845 raft_consensus.cc:2243] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.507941 29845 raft_consensus.cc:2272] T e12aacff6f754dfaa669e9651cce7dd9 P 99db41e57175426b9ceb11625b4455d9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.510028 29845 tablet_replica.cc:333] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.510576 29845 raft_consensus.cc:2243] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.511004 29845 raft_consensus.cc:2272] T 4ac46831fcf945d194fd7677ada23856 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.512790 29845 tablet_replica.cc:333] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.513298 29845 raft_consensus.cc:2243] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.514096 29845 raft_consensus.cc:2272] T d2fcd7006bc840a0bb5ea27365793f86 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.516742 29845 tablet_replica.cc:333] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.517433 29845 raft_consensus.cc:2243] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.518472 29845 raft_consensus.cc:2272] T 75c82b1c500f4214aef5d8ed2567f780 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.521190 29845 tablet_replica.cc:333] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.521942 29845 raft_consensus.cc:2243] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.522517 29845 raft_consensus.cc:2272] T ea38fed35e6549b3917d485f9af88382 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.525377 29845 tablet_replica.cc:333] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.526109 29845 raft_consensus.cc:2243] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.526638 29845 raft_consensus.cc:2272] T 7b68394604b04367a2e1169ce7623abc P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.530699 29845 tablet_replica.cc:333] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.531243 29845 raft_consensus.cc:2243] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.532055 29845 raft_consensus.cc:2272] T f0698605f5af41fcbc332b4dc8857606 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.534089 29845 tablet_replica.cc:333] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.534639 29845 raft_consensus.cc:2243] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.535398 29845 raft_consensus.cc:2272] T b21f93c356074f488a5be203ff0b1988 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.537204 29845 tablet_replica.cc:333] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.537717 29845 raft_consensus.cc:2243] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.538143 29845 raft_consensus.cc:2272] T 0c2693326c6b49989473c3de8c70d40a P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.540161 29845 tablet_replica.cc:333] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.540828 29845 raft_consensus.cc:2243] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.541505 29845 raft_consensus.cc:2272] T 8c77ef2e07ff418b996e1c72415228d6 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.543316 29845 tablet_replica.cc:333] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.543745 29845 raft_consensus.cc:2243] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.544435 29845 raft_consensus.cc:2272] T b678720774394be3899ea14ff99628da P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.546165 29845 tablet_replica.cc:333] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.546676 29845 raft_consensus.cc:2243] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.547048 29845 raft_consensus.cc:2272] T 0aa7e52e7c5b4039a78f136142693567 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.548940 29845 tablet_replica.cc:333] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.549386 29845 raft_consensus.cc:2243] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.549726 29845 raft_consensus.cc:2272] T 052922f90cab431dba48fb4991dee3fb P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.551563 29845 tablet_replica.cc:333] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.552235 29845 raft_consensus.cc:2243] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.552596 29845 raft_consensus.cc:2272] T 864b9e85f77d4ecc8726f6d624b5780c P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.554936 29845 tablet_replica.cc:333] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.555420 29845 raft_consensus.cc:2243] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.556376 29845 raft_consensus.cc:2272] T a38ba9e3e56242779781f03654f3dfd3 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.556562  6942 consensus_peers.cc:597] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.558591 29845 tablet_replica.cc:333] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.559345 29845 raft_consensus.cc:2243] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.559855 29845 raft_consensus.cc:2272] T c2acfad3040b40b69d4e0262179cdf6e P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.562000  6942 consensus_peers.cc:597] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.562700 29845 tablet_replica.cc:333] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.563315 29845 raft_consensus.cc:2243] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:09.563445  7016 consensus_peers.cc:597] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.563786 29845 raft_consensus.cc:2272] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.565623 29845 tablet_replica.cc:333] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.566102 29845 raft_consensus.cc:2243] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.566875 29845 raft_consensus.cc:2272] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 99db41e57175426b9ceb11625b4455d9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.568934 29845 tablet_replica.cc:333] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9: stopping tablet replica
I20260709 12:28:09.569471 29845 raft_consensus.cc:2243] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.569883 29845 raft_consensus.cc:2272] T f3ed0915d1da4514bc4c51eddfcf5e31 P 99db41e57175426b9ceb11625b4455d9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.594432  6942 consensus_peers.cc:597] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.618952  6942 consensus_peers.cc:597] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.630380  6942 consensus_peers.cc:597] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.646672  7016 consensus_peers.cc:597] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:34951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.653196 29845 tablet_server.cc:196] TabletServer@127.29.37.66:0 shutdown complete.
W20260709 12:28:09.711419  7016 consensus_peers.cc:597] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:34951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.721184 29845 tablet_server.cc:179] TabletServer@127.29.37.67:0 shutting down...
W20260709 12:28:09.754719  7016 consensus_peers.cc:597] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:34951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.763904 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:09.764899 29845 tablet_replica.cc:333] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.765925 29845 raft_consensus.cc:2243] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.766561 29845 raft_consensus.cc:2272] T 07ecb64e39814c7a961ebb058d596cbc P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.768566 29845 tablet_replica.cc:333] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.769119 29845 raft_consensus.cc:2243] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.769518 29845 raft_consensus.cc:2272] T 5df4f6356b53461e89321a089ab9b65f P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.771397 29845 tablet_replica.cc:333] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.771867 29845 raft_consensus.cc:2243] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.772228 29845 raft_consensus.cc:2272] T 22a6861b97e443898d98ede72c787513 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.773916 29845 tablet_replica.cc:333] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.774473 29845 raft_consensus.cc:2243] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.774888 29845 raft_consensus.cc:2272] T 580547b3f100485eba38ef98536bbc93 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.776731 29845 tablet_replica.cc:333] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.777266 29845 raft_consensus.cc:2243] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.778044 29845 raft_consensus.cc:2272] T 70fa5eb82f684b449d82176587520e4d P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.780073 29845 tablet_replica.cc:333] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.780725 29845 raft_consensus.cc:2243] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.781080 29845 raft_consensus.cc:2272] T c92afb4b5d234fd48fe0f24ec262964a P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.782946 29845 tablet_replica.cc:333] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.783466 29845 raft_consensus.cc:2243] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.784144 29845 raft_consensus.cc:2272] T a690e13ef48d439cb389b897c6b9fcc6 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.785974 29845 tablet_replica.cc:333] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.786571 29845 raft_consensus.cc:2243] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.787320 29845 raft_consensus.cc:2272] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.789104 29845 tablet_replica.cc:333] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.789590 29845 raft_consensus.cc:2243] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.790009 29845 raft_consensus.cc:2272] T 4d22d1742e5946998079e448d9ab49ef P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.792373 29845 tablet_replica.cc:333] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.793073 29845 raft_consensus.cc:2243] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.793597 29845 raft_consensus.cc:2272] T 5257ef3bead24ca48d68d7b8415e116f P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.795783 29845 tablet_replica.cc:333] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.796317 29845 raft_consensus.cc:2243] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.796706 29845 raft_consensus.cc:2272] T efdd38fdcb5f4e2198f1180e3fe51a10 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.798511 29845 tablet_replica.cc:333] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.799060 29845 raft_consensus.cc:2243] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.799448 29845 raft_consensus.cc:2272] T e12aacff6f754dfaa669e9651cce7dd9 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.801348 29845 tablet_replica.cc:333] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.802086 29845 raft_consensus.cc:2243] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.803035 29845 raft_consensus.cc:2272] T 4ac46831fcf945d194fd7677ada23856 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.805450 29845 tablet_replica.cc:333] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.805976 29845 raft_consensus.cc:2243] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.806800 29845 raft_consensus.cc:2272] T 86c78f0ec1af4414a7e52c69f19b6c27 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.809096 29845 tablet_replica.cc:333] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.809679 29845 raft_consensus.cc:2243] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.810141 29845 raft_consensus.cc:2272] T 8203d585d06840daadbcfbb32fb2f822 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.812124 29845 tablet_replica.cc:333] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.812691 29845 raft_consensus.cc:2243] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.813474 29845 raft_consensus.cc:2272] T bfbf51a7748b461686ff7db762bba509 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.815410 29845 tablet_replica.cc:333] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.815974 29845 raft_consensus.cc:2243] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.816577 29845 raft_consensus.cc:2272] T 93204a5a0d0742ad9cd21a8a280af392 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.819119 29845 tablet_replica.cc:333] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.819736 29845 raft_consensus.cc:2243] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.820218 29845 raft_consensus.cc:2272] T d82cf9bc8bd54dc08a7e03270cacde99 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.822108 29845 tablet_replica.cc:333] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.822677 29845 raft_consensus.cc:2243] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.823091 29845 raft_consensus.cc:2272] T 02d86f9e2d5d4ca195483d90646330a4 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.824883 29845 tablet_replica.cc:333] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.825408 29845 raft_consensus.cc:2243] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.825810 29845 raft_consensus.cc:2272] T 46b2ca102c7b4575a89399cac44ee866 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.827702 29845 tablet_replica.cc:333] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.828315 29845 raft_consensus.cc:2243] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.828713 29845 raft_consensus.cc:2272] T d2fcd7006bc840a0bb5ea27365793f86 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.830755 29845 tablet_replica.cc:333] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.831297 29845 raft_consensus.cc:2243] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.831835 29845 raft_consensus.cc:2272] T 75c82b1c500f4214aef5d8ed2567f780 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.834313 29845 tablet_replica.cc:333] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.835995 29845 raft_consensus.cc:2243] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.836859 29845 raft_consensus.cc:2272] T 052922f90cab431dba48fb4991dee3fb P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.838052  6942 consensus_peers.cc:597] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:34951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.839543 29845 tablet_replica.cc:333] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.840345 29845 raft_consensus.cc:2243] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.840876 29845 raft_consensus.cc:2272] T 9875aff5560f4adc82ed2881b0719c77 P c78bd8036f7040b99af7c501182cbe21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.843321 29845 tablet_replica.cc:333] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.844164 29845 raft_consensus.cc:2243] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.844660 29845 raft_consensus.cc:2272] T cb45dbc587b34cb7843e48180e0219c3 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.847358 29845 tablet_replica.cc:333] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
W20260709 12:28:09.847992  7016 consensus_peers.cc:597] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:34951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.848116 29845 raft_consensus.cc:2243] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.848826 29845 raft_consensus.cc:2272] T 04e9253071944b29a6116b8d9f64ef3a P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.851509 29845 tablet_replica.cc:333] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
W20260709 12:28:09.851531  7013 consensus_peers.cc:597] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.852140  6942 consensus_peers.cc:597] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:34951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:09.852339  7013 consensus_peers.cc:597] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.852411 29845 raft_consensus.cc:2243] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:28:09.853116  7013 consensus_peers.cc:597] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.853624 29845 raft_consensus.cc:2272] T 864b9e85f77d4ecc8726f6d624b5780c P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.856462 29845 tablet_replica.cc:333] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.857052 29845 raft_consensus.cc:2243] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:28:09.857782  7013 consensus_peers.cc:597] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.858116 29845 raft_consensus.cc:2272] T c2acfad3040b40b69d4e0262179cdf6e P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.859052  7013 consensus_peers.cc:597] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.860272 29845 tablet_replica.cc:333] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.860865 29845 raft_consensus.cc:2243] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.861286 29845 raft_consensus.cc:2272] T 718060ceb1df4264ac14b8e4c8f9c007 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.863310 29845 tablet_replica.cc:333] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.863845 29845 raft_consensus.cc:2243] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.864562 29845 raft_consensus.cc:2272] T e7eee84ade114ccaa536076ae9b04ac2 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.866495 29845 tablet_replica.cc:333] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.867022 29845 raft_consensus.cc:2243] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.867470 29845 raft_consensus.cc:2272] T 122d63cf583043458b719df07a6a694d P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.869334 29845 tablet_replica.cc:333] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.869882 29845 raft_consensus.cc:2243] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.870343 29845 raft_consensus.cc:2272] T f0698605f5af41fcbc332b4dc8857606 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.872260 29845 tablet_replica.cc:333] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.872797 29845 raft_consensus.cc:2243] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.873226 29845 raft_consensus.cc:2272] T b21f93c356074f488a5be203ff0b1988 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.875130 29845 tablet_replica.cc:333] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.875658 29845 raft_consensus.cc:2243] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.876062 29845 raft_consensus.cc:2272] T b678720774394be3899ea14ff99628da P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.877888 29845 tablet_replica.cc:333] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.878466 29845 raft_consensus.cc:2243] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.879202 29845 raft_consensus.cc:2272] T a2d766dbe21d42a0ab36d005d89289c9 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.881139 29845 tablet_replica.cc:333] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.881645 29845 raft_consensus.cc:2243] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.882045 29845 raft_consensus.cc:2272] T 251b19e52f0146aa9a9cfc985e2763b4 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.883919 29845 tablet_replica.cc:333] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.884434 29845 raft_consensus.cc:2243] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.885257 29845 raft_consensus.cc:2272] T 0aa7e52e7c5b4039a78f136142693567 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.887161 29845 tablet_replica.cc:333] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.887656 29845 raft_consensus.cc:2243] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.888046 29845 raft_consensus.cc:2272] T a85b20398cd74f5888b96dc53ab9179b P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.889735 29845 tablet_replica.cc:333] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.890372 29845 raft_consensus.cc:2243] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.891135 29845 raft_consensus.cc:2272] T 847f2ce68f10434cb350d63a6573b3bd P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.892961 29845 tablet_replica.cc:333] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.893440 29845 raft_consensus.cc:2243] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.893847 29845 raft_consensus.cc:2272] T e962d48b16794d78ad1a367540885945 P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.896060 29845 tablet_replica.cc:333] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.896653 29845 raft_consensus.cc:2243] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.897101 29845 raft_consensus.cc:2272] T 7b68394604b04367a2e1169ce7623abc P c78bd8036f7040b99af7c501182cbe21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.899456 29845 tablet_replica.cc:333] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.900063 29845 raft_consensus.cc:2243] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:09.900601 29845 raft_consensus.cc:2272] T 4e2b62ad0158424696dcdf73faa4f675 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.902695  7016 consensus_peers.cc:597] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Couldn't send request to peer 99db41e57175426b9ceb11625b4455d9. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:34951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.903167 29845 tablet_replica.cc:333] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.903944 29845 raft_consensus.cc:2243] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.904878 29845 raft_consensus.cc:2272] T ea38fed35e6549b3917d485f9af88382 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:09.906718 29845 tablet_replica.cc:333] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21: stopping tablet replica
I20260709 12:28:09.907176 29845 raft_consensus.cc:2243] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:09.907927 29845 raft_consensus.cc:2272] T f3ed0915d1da4514bc4c51eddfcf5e31 P c78bd8036f7040b99af7c501182cbe21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.937676  7013 consensus_peers.cc:597] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.973063 29845 tablet_server.cc:196] TabletServer@127.29.37.67:0 shutdown complete.
I20260709 12:28:10.045166 29845 tablet_server.cc:179] TabletServer@127.29.37.68:0 shutting down...
W20260709 12:28:10.056923  7013 consensus_peers.cc:597] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.67:41667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:10.070742  7013 consensus_peers.cc:597] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.67:41667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:10.076506  7013 consensus_peers.cc:597] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.67:41667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:10.093186 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:10.094210 29845 tablet_replica.cc:333] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.095012 29845 raft_consensus.cc:2243] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.095534 29845 raft_consensus.cc:2272] T 3b065e90800f40379b501a6238c6fdcd P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.098465 29845 tablet_replica.cc:333] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.099208 29845 raft_consensus.cc:2243] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.100311 29845 raft_consensus.cc:2272] T 5df4f6356b53461e89321a089ab9b65f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:10.101414  7013 consensus_peers.cc:597] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 -> Peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Couldn't send request to peer c78bd8036f7040b99af7c501182cbe21. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.67:41667: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:10.103070 29845 tablet_replica.cc:333] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.104079 29845 raft_consensus.cc:2243] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.104836 29845 raft_consensus.cc:2272] T 07ecb64e39814c7a961ebb058d596cbc P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.107445 29845 tablet_replica.cc:333] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.108078 29845 raft_consensus.cc:2243] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.108558 29845 raft_consensus.cc:2272] T f932aaa29a8443d4b251dea7c9fc0f54 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.110730 29845 tablet_replica.cc:333] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.111363 29845 raft_consensus.cc:2243] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.112161 29845 raft_consensus.cc:2272] T c92afb4b5d234fd48fe0f24ec262964a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.114167 29845 tablet_replica.cc:333] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.114898 29845 raft_consensus.cc:2243] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.115334 29845 raft_consensus.cc:2272] T cd4c913d08ea42b09038a2a80ecfa3b6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.117528 29845 tablet_replica.cc:333] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.118112 29845 raft_consensus.cc:2243] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.118605 29845 raft_consensus.cc:2272] T e381f83716b64b7982f5f0c9c63d348f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.120105  7859 raft_consensus.cc:493] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:28:10.120599  7859 raft_consensus.cc:515] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } } peers { permanent_uuid: "99db41e57175426b9ceb11625b4455d9" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 34951 } }
I20260709 12:28:10.121340 29845 tablet_replica.cc:333] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.122298 29845 raft_consensus.cc:2243] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.122850 29845 raft_consensus.cc:2272] T 2dfa9e71770a4e3dac4348d84a9cc2d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.125885 29845 tablet_replica.cc:333] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.125540  7859 leader_election.cc:290] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171), 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951)
I20260709 12:28:10.126827 29845 raft_consensus.cc:2243] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:10.127796  7016 leader_election.cc:336] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99db41e57175426b9ceb11625b4455d9 (127.29.37.66:34951): Network error: Client connection negotiation failed: client connection to 127.29.37.66:34951: connect: Connection refused (error 111)
W20260709 12:28:10.128382  7013 leader_election.cc:336] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111)
I20260709 12:28:10.128470 29845 raft_consensus.cc:2272] T 70fa5eb82f684b449d82176587520e4d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.128795  7013 leader_election.cc:304] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, 99db41e57175426b9ceb11625b4455d9
I20260709 12:28:10.129522  7859 raft_consensus.cc:2749] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:10.132488 29845 tablet_replica.cc:333] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.133462 29845 raft_consensus.cc:2243] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.134528 29845 raft_consensus.cc:2272] T e11f6b3dd5fe466d984f796948818c0d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.136839 29845 tablet_replica.cc:333] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.137444 29845 raft_consensus.cc:2243] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.138299 29845 raft_consensus.cc:2272] T 4d22d1742e5946998079e448d9ab49ef P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.140334 29845 tablet_replica.cc:333] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.140910 29845 raft_consensus.cc:2243] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.141345 29845 raft_consensus.cc:2272] T 8ab076c84814416690df2fe770a98aa0 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.143311 29845 tablet_replica.cc:333] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.144086 29845 raft_consensus.cc:2243] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.144905 29845 raft_consensus.cc:2272] T a85b20398cd74f5888b96dc53ab9179b P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.146823 29845 tablet_replica.cc:333] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.147353 29845 raft_consensus.cc:2243] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.147742 29845 raft_consensus.cc:2272] T 36d0e6da2d6d4fa6abb2757dff7de498 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.149516 29845 tablet_replica.cc:333] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.150041 29845 raft_consensus.cc:2243] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.150851 29845 raft_consensus.cc:2272] T 8203d585d06840daadbcfbb32fb2f822 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.152825 29845 tablet_replica.cc:333] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.153354 29845 raft_consensus.cc:2243] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.153818 29845 raft_consensus.cc:2272] T 86c78f0ec1af4414a7e52c69f19b6c27 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.155742 29845 tablet_replica.cc:333] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.156273 29845 raft_consensus.cc:2243] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.156658 29845 raft_consensus.cc:2272] T bfbf51a7748b461686ff7db762bba509 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.158511 29845 tablet_replica.cc:333] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.159003 29845 raft_consensus.cc:2243] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.159780 29845 raft_consensus.cc:2272] T 93204a5a0d0742ad9cd21a8a280af392 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.161742 29845 tablet_replica.cc:333] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.162341 29845 raft_consensus.cc:2243] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.163335 29845 raft_consensus.cc:2272] T 8dc898afa9a94274850432d379c5203e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.165674 29845 tablet_replica.cc:333] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.166172 29845 raft_consensus.cc:2243] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.166605 29845 raft_consensus.cc:2272] T afde26a33c5c45cfb6b2087601e4ce01 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.168440 29845 tablet_replica.cc:333] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.168948 29845 raft_consensus.cc:2243] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.169332 29845 raft_consensus.cc:2272] T 75c82b1c500f4214aef5d8ed2567f780 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.171372 29845 tablet_replica.cc:333] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.171854 29845 raft_consensus.cc:2243] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.172267 29845 raft_consensus.cc:2272] T 9875aff5560f4adc82ed2881b0719c77 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.175060 29845 tablet_replica.cc:333] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.175812 29845 raft_consensus.cc:2243] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.176388 29845 raft_consensus.cc:2272] T 052922f90cab431dba48fb4991dee3fb P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.178984 29845 tablet_replica.cc:333] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.179553 29845 raft_consensus.cc:2243] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.180018 29845 raft_consensus.cc:2272] T 864b9e85f77d4ecc8726f6d624b5780c P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.182045 29845 tablet_replica.cc:333] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.182639 29845 raft_consensus.cc:2243] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.183099 29845 raft_consensus.cc:2272] T a38ba9e3e56242779781f03654f3dfd3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.185065 29845 tablet_replica.cc:333] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.185634 29845 raft_consensus.cc:2243] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.186050 29845 raft_consensus.cc:2272] T c2acfad3040b40b69d4e0262179cdf6e P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.187899 29845 tablet_replica.cc:333] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.188457 29845 raft_consensus.cc:2243] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.188889 29845 raft_consensus.cc:2272] T f0698605f5af41fcbc332b4dc8857606 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.190742 29845 tablet_replica.cc:333] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.191285 29845 raft_consensus.cc:2243] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.191691 29845 raft_consensus.cc:2272] T b21f93c356074f488a5be203ff0b1988 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.193765 29845 tablet_replica.cc:333] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.194366 29845 raft_consensus.cc:2243] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.195182 29845 raft_consensus.cc:2272] T 0c2693326c6b49989473c3de8c70d40a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.197187 29845 tablet_replica.cc:333] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.197345  7977 raft_consensus.cc:493] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38b3b964f82644c7b654e8042953a74e)
I20260709 12:28:10.197788 29845 raft_consensus.cc:2243] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.197827  7977 raft_consensus.cc:515] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c78bd8036f7040b99af7c501182cbe21" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 41667 } } peers { permanent_uuid: "3fb7d0187b544dfdb8487ecb2ff24a76" member_type: VOTER last_known_addr { host: "127.29.37.68" port: 41389 } } peers { permanent_uuid: "38b3b964f82644c7b654e8042953a74e" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 45171 } }
I20260709 12:28:10.198434 29845 raft_consensus.cc:2272] T b678720774394be3899ea14ff99628da P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.199827  7977 leader_election.cc:290] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667), 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171)
I20260709 12:28:10.201182 29845 tablet_replica.cc:333] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.201990 29845 raft_consensus.cc:2243] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.202544 29845 raft_consensus.cc:2272] T 8c77ef2e07ff418b996e1c72415228d6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:10.203797  7013 leader_election.cc:336] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c78bd8036f7040b99af7c501182cbe21 (127.29.37.67:41667): Network error: Client connection negotiation failed: client connection to 127.29.37.67:41667: connect: Connection refused (error 111)
W20260709 12:28:10.204540  7013 leader_election.cc:336] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 38b3b964f82644c7b654e8042953a74e (127.29.37.65:45171): Network error: Client connection negotiation failed: client connection to 127.29.37.65:45171: connect: Connection refused (error 111)
I20260709 12:28:10.204931  7013 leader_election.cc:304] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb7d0187b544dfdb8487ecb2ff24a76; no voters: 38b3b964f82644c7b654e8042953a74e, c78bd8036f7040b99af7c501182cbe21
I20260709 12:28:10.205284 29845 tablet_replica.cc:333] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.205622  7977 raft_consensus.cc:2749] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:10.206081 29845 raft_consensus.cc:2243] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.206686 29845 raft_consensus.cc:2272] T e7eee84ade114ccaa536076ae9b04ac2 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.208631 29845 tablet_replica.cc:333] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.209179 29845 raft_consensus.cc:2243] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.210115 29845 raft_consensus.cc:2272] T 5b8fc940ac6a48f0ab40f16b9a7b9ca6 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.212347 29845 tablet_replica.cc:333] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.212898 29845 raft_consensus.cc:2243] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.213337 29845 raft_consensus.cc:2272] T b2429fdc971b4c048fbc5f3ce3b6bc2f P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.215368 29845 tablet_replica.cc:333] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.215920 29845 raft_consensus.cc:2243] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.216362 29845 raft_consensus.cc:2272] T efdd38fdcb5f4e2198f1180e3fe51a10 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.218389 29845 tablet_replica.cc:333] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.218922 29845 raft_consensus.cc:2243] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.219835 29845 raft_consensus.cc:2272] T 122d63cf583043458b719df07a6a694d P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.221781 29845 tablet_replica.cc:333] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.222388 29845 raft_consensus.cc:2243] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.222838 29845 raft_consensus.cc:2272] T a2d766dbe21d42a0ab36d005d89289c9 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.224689 29845 tablet_replica.cc:333] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.225242 29845 raft_consensus.cc:2243] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.226042 29845 raft_consensus.cc:2272] T 251b19e52f0146aa9a9cfc985e2763b4 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.228135 29845 tablet_replica.cc:333] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.228657 29845 raft_consensus.cc:2243] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.229111 29845 raft_consensus.cc:2272] T 7c55f4a15fa641de92c4922bc65c06be P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.231241 29845 tablet_replica.cc:333] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.231760 29845 raft_consensus.cc:2243] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.232198 29845 raft_consensus.cc:2272] T e962d48b16794d78ad1a367540885945 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.234089 29845 tablet_replica.cc:333] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.234651 29845 raft_consensus.cc:2243] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.235468 29845 raft_consensus.cc:2272] T 4e2b62ad0158424696dcdf73faa4f675 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.237552 29845 tablet_replica.cc:333] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.238123 29845 raft_consensus.cc:2243] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.238610 29845 raft_consensus.cc:2272] T ea38fed35e6549b3917d485f9af88382 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.240535 29845 tablet_replica.cc:333] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.241044 29845 raft_consensus.cc:2243] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:10.241515 29845 raft_consensus.cc:2272] T cb45dbc587b34cb7843e48180e0219c3 P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.243541 29845 tablet_replica.cc:333] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76: stopping tablet replica
I20260709 12:28:10.244025 29845 raft_consensus.cc:2243] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.244822 29845 raft_consensus.cc:2272] T 04e9253071944b29a6116b8d9f64ef3a P 3fb7d0187b544dfdb8487ecb2ff24a76 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.313735 29845 tablet_server.cc:196] TabletServer@127.29.37.68:0 shutdown complete.
I20260709 12:28:10.377194 29845 master.cc:562] Master@127.29.37.126:36231 shutting down...
I20260709 12:28:10.401784 29845 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:10.402426 29845 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:10.402777 29845 tablet_replica.cc:333] T 00000000000000000000000000000000 P 66ecd0b72fc14e0b8a155ba9063e4623: stopping tablet replica
I20260709 12:28:10.447418 29845 master.cc:584] Master@127.29.37.126:36231 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (28333 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:28:10.532820 29845 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.29.37.126:37501
I20260709 12:28:10.533967 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:10.539800  7980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:10.539830  7981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:10.540907  7983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:10.541823 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:10.543020 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:10.543246 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:10.543406 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600090543387 us; error 0 us; skew 500 ppm
I20260709 12:28:10.543951 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:10.546342 29845 webserver.cc:533] Webserver started at http://127.29.37.126:38369/ using document root <none> and password file <none>
I20260709 12:28:10.546824 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:10.547024 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:10.547358 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:10.548445 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/master-0-root/instance:
uuid: "267aa51e8905414787e6c01124ee21d4"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:10.552767 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:28:10.555966  7988 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.556866 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:28:10.557104 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/master-0-root
uuid: "267aa51e8905414787e6c01124ee21d4"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:10.557338 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:10.574563 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:10.575682 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:10.613209 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.126:37501
I20260709 12:28:10.613324  8039 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.126:37501 every 8 connection(s)
I20260709 12:28:10.617266  8040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:10.628541  8040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4: Bootstrap starting.
I20260709 12:28:10.633483  8040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:10.637519  8040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4: No bootstrap required, opened a new log
I20260709 12:28:10.640226  8040 raft_consensus.cc:359] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267aa51e8905414787e6c01124ee21d4" member_type: VOTER }
I20260709 12:28:10.640615  8040 raft_consensus.cc:385] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:10.640856  8040 raft_consensus.cc:740] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267aa51e8905414787e6c01124ee21d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:10.641508  8040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267aa51e8905414787e6c01124ee21d4" member_type: VOTER }
I20260709 12:28:10.641980  8040 raft_consensus.cc:399] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:10.642211  8040 raft_consensus.cc:493] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:10.642486  8040 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:10.646806  8040 raft_consensus.cc:515] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267aa51e8905414787e6c01124ee21d4" member_type: VOTER }
I20260709 12:28:10.647296  8040 leader_election.cc:304] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 267aa51e8905414787e6c01124ee21d4; no voters: 
I20260709 12:28:10.648392  8040 leader_election.cc:290] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:10.648787  8043 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:10.650107  8043 raft_consensus.cc:697] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 1 LEADER]: Becoming Leader. State: Replica: 267aa51e8905414787e6c01124ee21d4, State: Running, Role: LEADER
I20260709 12:28:10.650787  8043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267aa51e8905414787e6c01124ee21d4" member_type: VOTER }
I20260709 12:28:10.651353  8040 sys_catalog.cc:565] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:10.656973  8045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 267aa51e8905414787e6c01124ee21d4. Latest consensus state: current_term: 1 leader_uuid: "267aa51e8905414787e6c01124ee21d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267aa51e8905414787e6c01124ee21d4" member_type: VOTER } }
I20260709 12:28:10.657550  8045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:10.658068  8044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "267aa51e8905414787e6c01124ee21d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267aa51e8905414787e6c01124ee21d4" member_type: VOTER } }
I20260709 12:28:10.658694  8044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:10.660161  8051 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:10.667769  8051 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:10.668752 29845 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:10.676323  8051 catalog_manager.cc:1383] Generated new cluster ID: e0e709b91a0840a29758f7f70fca2324
I20260709 12:28:10.676549  8051 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:10.702047  8051 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:10.703401  8051 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:10.720582  8051 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4: Generated new TSK 0
I20260709 12:28:10.721133  8051 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:10.735369 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:10.741199  8061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:10.742426  8062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:10.746246  8064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:10.746764 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:10.750062 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:10.750352 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:10.750561 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600090750541 us; error 0 us; skew 500 ppm
I20260709 12:28:10.751143 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:10.753538 29845 webserver.cc:533] Webserver started at http://127.29.37.65:38169/ using document root <none> and password file <none>
I20260709 12:28:10.754057 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:10.754280 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:10.754557 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:10.755609 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root/instance:
uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:10.759963 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:28:10.763433  8069 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.764169 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:28:10.764462 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:10.764768 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:10.780052 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:10.781209 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:10.782739 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:10.784941 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:10.785166 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.785400 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:10.785583 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.824360 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:35385
I20260709 12:28:10.824473  8131 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:35385 every 8 connection(s)
I20260709 12:28:10.829571 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:10.837347  8136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:10.839360  8137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:10.840546  8139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:10.841653 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:10.842950 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:10.843168 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:10.843355 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600090843336 us; error 0 us; skew 500 ppm
I20260709 12:28:10.843915 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:10.846573 29845 webserver.cc:533] Webserver started at http://127.29.37.66:42567/ using document root <none> and password file <none>
I20260709 12:28:10.847224 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:10.847456 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:10.847752 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:10.848181  8132 heartbeater.cc:344] Connected to a master server at 127.29.37.126:37501
I20260709 12:28:10.848619  8132 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:10.849167 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-1-root/instance:
uuid: "712be1df6e174cd9928a200ac0a17e59"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:10.849514  8132 heartbeater.cc:507] Master 127.29.37.126:37501 requested a full tablet report, sending...
I20260709 12:28:10.851802  8005 ts_manager.cc:194] Registered new tserver with Master: 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385)
I20260709 12:28:10.853829  8005 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59190
I20260709 12:28:10.854508 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:28:10.857931  8144 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.858762 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:28:10.859056 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-1-root
uuid: "712be1df6e174cd9928a200ac0a17e59"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:10.859333 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:10.871434 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:10.872498 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:10.873886 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:10.876020 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:10.876235 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.876468 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:10.876636 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.914937 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.66:37049
I20260709 12:28:10.915051  8206 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.66:37049 every 8 connection(s)
I20260709 12:28:10.919641 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:10.926647  8210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:10.927994  8211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:10.933063  8213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:10.934577 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:10.934790  8207 heartbeater.cc:344] Connected to a master server at 127.29.37.126:37501
I20260709 12:28:10.935292  8207 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:10.935628 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:10.935911 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:10.936020  8207 heartbeater.cc:507] Master 127.29.37.126:37501 requested a full tablet report, sending...
I20260709 12:28:10.936100 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600090936083 us; error 0 us; skew 500 ppm
I20260709 12:28:10.936992 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:10.938553  8005 ts_manager.cc:194] Registered new tserver with Master: 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:10.939766 29845 webserver.cc:533] Webserver started at http://127.29.37.67:36251/ using document root <none> and password file <none>
I20260709 12:28:10.940465 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:10.940428  8005 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59206
I20260709 12:28:10.940939 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:10.941291 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:10.942857 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-2-root/instance:
uuid: "72fb489f0a054602bb946316bc7c187e"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:10.947314 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:28:10.950547  8218 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.951346 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:10.951601 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-2-root
uuid: "72fb489f0a054602bb946316bc7c187e"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:10.951848 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:10.962445 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:10.963470 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:10.964797 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:10.967031 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:10.967259 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:10.967494 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:10.967665 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:11.006024 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.67:35273
I20260709 12:28:11.006130  8280 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.67:35273 every 8 connection(s)
I20260709 12:28:11.023042  8281 heartbeater.cc:344] Connected to a master server at 127.29.37.126:37501
I20260709 12:28:11.023451  8281 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:11.024119  8281 heartbeater.cc:507] Master 127.29.37.126:37501 requested a full tablet report, sending...
I20260709 12:28:11.026122  8005 ts_manager.cc:194] Registered new tserver with Master: 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.027118 29845 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01790374s
I20260709 12:28:11.027784  8005 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59210
I20260709 12:28:11.050869  8005 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59226:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:11.053766  8005 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:11.135314  8169 tablet_service.cc:1511] Processing CreateTablet for tablet abcb28adc6ec45029601c575e7640ae2 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:11.135792  8170 tablet_service.cc:1511] Processing CreateTablet for tablet 86e376d584f94d5c84247a21cb2b0d76 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:11.135408  8172 tablet_service.cc:1511] Processing CreateTablet for tablet 7f90b970828c470d9efd8522dee3ddde (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:11.136969  8169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abcb28adc6ec45029601c575e7640ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.137902  8170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e376d584f94d5c84247a21cb2b0d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.140352  8171 tablet_service.cc:1511] Processing CreateTablet for tablet 663664e06ddc4e86a7055c8a134940cc (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:11.143718  8171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663664e06ddc4e86a7055c8a134940cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.143560  8096 tablet_service.cc:1511] Processing CreateTablet for tablet 663664e06ddc4e86a7055c8a134940cc (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:11.144914  8096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663664e06ddc4e86a7055c8a134940cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.147325  8095 tablet_service.cc:1511] Processing CreateTablet for tablet 86e376d584f94d5c84247a21cb2b0d76 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:11.148619  8095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e376d584f94d5c84247a21cb2b0d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.151763  8093 tablet_service.cc:1511] Processing CreateTablet for tablet e4d449e67e41460db81d4df8bff90f65 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:11.155395  8094 tablet_service.cc:1511] Processing CreateTablet for tablet abcb28adc6ec45029601c575e7640ae2 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:11.141848  8097 tablet_service.cc:1511] Processing CreateTablet for tablet 7f90b970828c470d9efd8522dee3ddde (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:11.157523  8172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f90b970828c470d9efd8522dee3ddde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.159682  8243 tablet_service.cc:1511] Processing CreateTablet for tablet abcb28adc6ec45029601c575e7640ae2 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:28:11.161211  8244 tablet_service.cc:1511] Processing CreateTablet for tablet 86e376d584f94d5c84247a21cb2b0d76 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:28:11.161310  8246 tablet_service.cc:1511] Processing CreateTablet for tablet 7f90b970828c470d9efd8522dee3ddde (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:28:11.161487  8245 tablet_service.cc:1511] Processing CreateTablet for tablet 663664e06ddc4e86a7055c8a134940cc (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:28:11.167477  8093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4d449e67e41460db81d4df8bff90f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.170634  8243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abcb28adc6ec45029601c575e7640ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.170831  8097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f90b970828c470d9efd8522dee3ddde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.171296  8244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86e376d584f94d5c84247a21cb2b0d76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.174679  8245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663664e06ddc4e86a7055c8a134940cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.177449  8094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abcb28adc6ec45029601c575e7640ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.178603  8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f90b970828c470d9efd8522dee3ddde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.197237  8168 tablet_service.cc:1511] Processing CreateTablet for tablet e4d449e67e41460db81d4df8bff90f65 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:11.198654  8168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4d449e67e41460db81d4df8bff90f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.212958  8242 tablet_service.cc:1511] Processing CreateTablet for tablet e4d449e67e41460db81d4df8bff90f65 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:28:11.214406  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4d449e67e41460db81d4df8bff90f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.232537  8296 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:11.238193  8297 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.240634  8093 tablet_service.cc:1511] Processing CreateTablet for tablet 8da354781025485892cf19d483eb32a3 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:11.241973  8093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da354781025485892cf19d483eb32a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.241963  8171 tablet_service.cc:1511] Processing CreateTablet for tablet 8da354781025485892cf19d483eb32a3 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:11.243430  8171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da354781025485892cf19d483eb32a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.268993  8297 tablet_bootstrap.cc:654] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.272853  8298 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.285604  8297 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:11.276561  8296 tablet_bootstrap.cc:654] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.286161  8297 ts_tablet_manager.cc:1403] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.048s	user 0.004s	sys 0.020s
I20260709 12:28:11.288200  8093 tablet_service.cc:1511] Processing CreateTablet for tablet 5f751c6cc07d44d28d60a92e61400400 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:11.289556  8093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f751c6cc07d44d28d60a92e61400400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.292755  8246 tablet_service.cc:1511] Processing CreateTablet for tablet 5f751c6cc07d44d28d60a92e61400400 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:11.289254  8297 raft_consensus.cc:359] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.289479  8172 tablet_service.cc:1511] Processing CreateTablet for tablet 5f751c6cc07d44d28d60a92e61400400 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:28:11.294495  8297 raft_consensus.cc:385] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.294871  8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f751c6cc07d44d28d60a92e61400400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.295135  8172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f751c6cc07d44d28d60a92e61400400. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.297093  8298 tablet_bootstrap.cc:654] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.294814  8297 raft_consensus.cc:740] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.298661  8297 consensus_queue.cc:260] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.310498  8094 tablet_service.cc:1511] Processing CreateTablet for tablet 27a7c938363946bf9efcf432ea6d935a (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:11.289515  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 8da354781025485892cf19d483eb32a3 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:28:11.311184  8171 tablet_service.cc:1511] Processing CreateTablet for tablet 27a7c938363946bf9efcf432ea6d935a (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:11.311828  8094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a7c938363946bf9efcf432ea6d935a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.312337  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8da354781025485892cf19d483eb32a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.315990  8243 tablet_service.cc:1511] Processing CreateTablet for tablet 27a7c938363946bf9efcf432ea6d935a (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:28:11.312479  8171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a7c938363946bf9efcf432ea6d935a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.317239  8243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27a7c938363946bf9efcf432ea6d935a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.317461  8298 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:11.317951  8298 ts_tablet_manager.cc:1403] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.045s	user 0.006s	sys 0.011s
I20260709 12:28:11.320977  8298 raft_consensus.cc:359] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.321658  8298 raft_consensus.cc:385] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.331929  8297 ts_tablet_manager.cc:1434] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 12:28:11.332726  8132 heartbeater.cc:499] Master 127.29.37.126:37501 was elected leader, sending a full tablet report...
I20260709 12:28:11.321954  8298 raft_consensus.cc:740] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.336731  8298 consensus_queue.cc:260] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.347168  8297 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.362581  8094 tablet_service.cc:1511] Processing CreateTablet for tablet 17bb19f5d45240ef8d5b1facd4c12e45 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:11.363929  8094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17bb19f5d45240ef8d5b1facd4c12e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.364795  8172 tablet_service.cc:1511] Processing CreateTablet for tablet 17bb19f5d45240ef8d5b1facd4c12e45 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:11.366498  8172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17bb19f5d45240ef8d5b1facd4c12e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.367036  8093 tablet_service.cc:1511] Processing CreateTablet for tablet 2face0b298ca4aa7a64cc288d6c5f6fa (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:11.368252  8093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2face0b298ca4aa7a64cc288d6c5f6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.379452  8168 tablet_service.cc:1511] Processing CreateTablet for tablet 2face0b298ca4aa7a64cc288d6c5f6fa (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:11.380678  8168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2face0b298ca4aa7a64cc288d6c5f6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.389209  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 17bb19f5d45240ef8d5b1facd4c12e45 (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:28:11.390988  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17bb19f5d45240ef8d5b1facd4c12e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.376803  8297 tablet_bootstrap.cc:654] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.393291  8281 heartbeater.cc:499] Master 127.29.37.126:37501 was elected leader, sending a full tablet report...
I20260709 12:28:11.397764  8296 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:11.398330  8296 ts_tablet_manager.cc:1403] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.166s	user 0.021s	sys 0.061s
I20260709 12:28:11.401391  8296 raft_consensus.cc:359] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.411589  8242 tablet_service.cc:1511] Processing CreateTablet for tablet 2face0b298ca4aa7a64cc288d6c5f6fa (DEFAULT_TABLE table=test-workload [id=986462d2c4904fe89cc9c3ce3336d614]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:28:11.412891  8242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2face0b298ca4aa7a64cc288d6c5f6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:11.402081  8296 raft_consensus.cc:385] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.413924  8296 raft_consensus.cc:740] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.414196  8298 ts_tablet_manager.cc:1434] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.096s	user 0.005s	sys 0.040s
I20260709 12:28:11.414875  8296 consensus_queue.cc:260] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.415447  8298 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.422289  8298 tablet_bootstrap.cc:654] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.425633  8297 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:11.426323  8297 ts_tablet_manager.cc:1403] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.079s	user 0.011s	sys 0.004s
I20260709 12:28:11.428755  8296 ts_tablet_manager.cc:1434] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.013s
I20260709 12:28:11.429443  8207 heartbeater.cc:499] Master 127.29.37.126:37501 was elected leader, sending a full tablet report...
I20260709 12:28:11.429842  8296 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:11.429661  8297 raft_consensus.cc:359] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.430517  8297 raft_consensus.cc:385] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.430858  8298 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:11.430811  8297 raft_consensus.cc:740] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.431502  8298 ts_tablet_manager.cc:1403] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.006s
I20260709 12:28:11.431892  8297 consensus_queue.cc:260] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.434600  8298 raft_consensus.cc:359] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.435389  8298 raft_consensus.cc:385] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.435664  8298 raft_consensus.cc:740] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.436496  8298 consensus_queue.cc:260] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.438710  8298 ts_tablet_manager.cc:1434] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:28:11.439574  8297 ts_tablet_manager.cc:1434] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:28:11.439735  8298 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.440598  8297 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.446789  8298 tablet_bootstrap.cc:654] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.437161  8296 tablet_bootstrap.cc:654] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.451813  8298 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:11.453029  8298 ts_tablet_manager.cc:1403] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.005s
I20260709 12:28:11.456460  8298 raft_consensus.cc:359] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.457260  8297 tablet_bootstrap.cc:654] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.457491  8298 raft_consensus.cc:385] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.458079  8298 raft_consensus.cc:740] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.459334  8298 consensus_queue.cc:260] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.462280  8298 ts_tablet_manager.cc:1434] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 12:28:11.463588  8298 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.466053  8296 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:11.466706  8296 ts_tablet_manager.cc:1403] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.007s
I20260709 12:28:11.469863  8298 tablet_bootstrap.cc:654] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.469686  8296 raft_consensus.cc:359] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.470461  8296 raft_consensus.cc:385] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.470793  8296 raft_consensus.cc:740] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.471591  8296 consensus_queue.cc:260] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.473901  8296 ts_tablet_manager.cc:1434] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:28:11.475183  8296 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:11.482621  8296 tablet_bootstrap.cc:654] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.482621  8298 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:11.483289  8305 raft_consensus.cc:493] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.483866  8305 raft_consensus.cc:515] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.490622  8305 leader_election.cc:290] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.491145  8297 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:11.491804  8297 ts_tablet_manager.cc:1403] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.025s
I20260709 12:28:11.495754  8298 ts_tablet_manager.cc:1403] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.007s
I20260709 12:28:11.495687  8297 raft_consensus.cc:359] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.496483  8297 raft_consensus.cc:385] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.496805  8297 raft_consensus.cc:740] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.497668  8297 consensus_queue.cc:260] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.502993  8298 raft_consensus.cc:359] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.503756  8298 raft_consensus.cc:385] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.504065  8298 raft_consensus.cc:740] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.504848  8298 consensus_queue.cc:260] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.507413  8297 ts_tablet_manager.cc:1434] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:28:11.514540  8297 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.517668  8298 ts_tablet_manager.cc:1434] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:28:11.518798  8298 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.525672  8305 raft_consensus.cc:493] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.526242  8305 raft_consensus.cc:515] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.527127  8297 tablet_bootstrap.cc:654] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.528520  8305 leader_election.cc:290] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.533632  8107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663664e06ddc4e86a7055c8a134940cc" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:11.533823  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e376d584f94d5c84247a21cb2b0d76" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:11.534659  8107 raft_consensus.cc:2468] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:11.534880  8106 raft_consensus.cc:2468] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:11.536443  8148 leader_election.cc:304] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:11.537264  8305 raft_consensus.cc:2804] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:11.537426  8148 leader_election.cc:304] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:11.537657  8305 raft_consensus.cc:493] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.538038  8305 raft_consensus.cc:3060] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.538498  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663664e06ddc4e86a7055c8a134940cc" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:11.539449  8256 raft_consensus.cc:2468] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:11.541100  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e376d584f94d5c84247a21cb2b0d76" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:11.541985  8255 raft_consensus.cc:2468] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:11.547509  8297 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:11.547520  8296 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:11.548221  8297 ts_tablet_manager.cc:1403] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.014s
I20260709 12:28:11.548249  8296 ts_tablet_manager.cc:1403] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.073s	user 0.016s	sys 0.048s
I20260709 12:28:11.548424  8305 raft_consensus.cc:515] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.551286  8297 raft_consensus.cc:359] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.551999  8297 raft_consensus.cc:385] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.552290  8297 raft_consensus.cc:740] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.552664  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663664e06ddc4e86a7055c8a134940cc" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:11.552780  8296 raft_consensus.cc:359] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.553109  8297 consensus_queue.cc:260] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.553642  8255 raft_consensus.cc:3060] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.553951  8296 raft_consensus.cc:385] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.554427  8296 raft_consensus.cc:740] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.555569  8297 ts_tablet_manager.cc:1434] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:28:11.555256  8296 consensus_queue.cc:260] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.556691  8297 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.558677  8298 tablet_bootstrap.cc:654] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.562527  8255 raft_consensus.cc:2468] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:11.563844  8146 leader_election.cc:304] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.565181  8315 raft_consensus.cc:2804] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:11.565688  8297 tablet_bootstrap.cc:654] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.568960  8305 leader_election.cc:290] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.569754  8315 raft_consensus.cc:493] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.570274  8315 raft_consensus.cc:3060] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.572319  8305 raft_consensus.cc:2804] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:11.572217  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663664e06ddc4e86a7055c8a134940cc" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:11.572942  8106 raft_consensus.cc:3060] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.579106  8305 raft_consensus.cc:697] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:11.580050  8106 raft_consensus.cc:2468] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:11.580000  8305 consensus_queue.cc:237] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.585023  8296 ts_tablet_manager.cc:1434] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.036s	user 0.003s	sys 0.016s
I20260709 12:28:11.586257  8296 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:11.587838  8297 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:11.588378  8297 ts_tablet_manager.cc:1403] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.004s
I20260709 12:28:11.589912  8315 raft_consensus.cc:515] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.591691  8297 raft_consensus.cc:359] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.592465  8297 raft_consensus.cc:385] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.592828  8298 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:11.592779  8297 raft_consensus.cc:740] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.593199  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e376d584f94d5c84247a21cb2b0d76" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:11.593554  8298 ts_tablet_manager.cc:1403] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.075s	user 0.011s	sys 0.009s
I20260709 12:28:11.593958  8255 raft_consensus.cc:3060] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.593909  8297 consensus_queue.cc:260] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.595412  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e376d584f94d5c84247a21cb2b0d76" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:11.595989  8106 raft_consensus.cc:3060] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.597425  8298 raft_consensus.cc:359] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.600782  8298 raft_consensus.cc:385] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.601141  8298 raft_consensus.cc:740] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.601758  8255 raft_consensus.cc:2468] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:11.600435  8296 tablet_bootstrap.cc:654] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.593318  8003 catalog_manager.cc:5697] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:11.602955  8146 leader_election.cc:304] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.603550  8297 ts_tablet_manager.cc:1434] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:28:11.603703  8106 raft_consensus.cc:2468] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:11.603925  8305 raft_consensus.cc:2804] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:11.604455  8305 raft_consensus.cc:697] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:11.605481  8315 leader_election.cc:290] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.605309  8305 consensus_queue.cc:237] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.602048  8298 consensus_queue.cc:260] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.609237  8298 ts_tablet_manager.cc:1434] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.005s
I20260709 12:28:11.610303  8298 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.612474  8297 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.617630  8296 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:11.618249  8296 ts_tablet_manager.cc:1403] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.000s
I20260709 12:28:11.620317  8297 tablet_bootstrap.cc:654] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.621553  8296 raft_consensus.cc:359] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.622438  8296 raft_consensus.cc:385] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.622820  8296 raft_consensus.cc:740] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.623760  8296 consensus_queue.cc:260] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.624377  8298 tablet_bootstrap.cc:654] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.626392  8296 ts_tablet_manager.cc:1434] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:11.627701  8296 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:11.632202  8303 raft_consensus.cc:493] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.632894  8303 raft_consensus.cc:515] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.636291  8296 tablet_bootstrap.cc:654] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.643276  8303 leader_election.cc:290] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:11.650534  8297 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:11.651264  8297 ts_tablet_manager.cc:1403] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.001s
I20260709 12:28:11.657022  8297 raft_consensus.cc:359] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.657970  8297 raft_consensus.cc:385] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.658355  8297 raft_consensus.cc:740] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.663327  8296 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:11.664032  8296 ts_tablet_manager.cc:1403] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.037s	user 0.024s	sys 0.005s
I20260709 12:28:11.667200  8296 raft_consensus.cc:359] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.668251  8296 raft_consensus.cc:385] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.667932  8297 consensus_queue.cc:260] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.663326  8298 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:11.669127  8298 ts_tablet_manager.cc:1403] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.031s
I20260709 12:28:11.669926  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d449e67e41460db81d4df8bff90f65" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59" is_pre_election: true
I20260709 12:28:11.670656  8182 raft_consensus.cc:2468] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 0.
I20260709 12:28:11.668663  8296 raft_consensus.cc:740] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.671856  8296 consensus_queue.cc:260] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.672144  8219 leader_election.cc:304] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.672049  8298 raft_consensus.cc:359] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.672891  8298 raft_consensus.cc:385] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.673154  8303 raft_consensus.cc:2804] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:11.673610  8303 raft_consensus.cc:493] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.673977  8303 raft_consensus.cc:3060] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.674875  8297 ts_tablet_manager.cc:1434] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.005s
I20260709 12:28:11.673172  8298 raft_consensus.cc:740] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.675985  8297 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.674916  8296 ts_tablet_manager.cc:1434] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
W20260709 12:28:11.676962  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:11.676692  8298 consensus_queue.cc:260] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.677601  8296 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:11.686697  8303 raft_consensus.cc:515] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.687357  8004 catalog_manager.cc:5697] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:11.690197  8297 tablet_bootstrap.cc:654] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.691668  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d449e67e41460db81d4df8bff90f65" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:11.692492  8182 raft_consensus.cc:3060] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.694072  8296 tablet_bootstrap.cc:654] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.709309  8303 leader_election.cc:290] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:11.715356  8303 raft_consensus.cc:493] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.715971  8303 raft_consensus.cc:515] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.728096  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d449e67e41460db81d4df8bff90f65" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:11.728950  8106 raft_consensus.cc:2468] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 0.
I20260709 12:28:11.731285  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d449e67e41460db81d4df8bff90f65" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:11.731994  8106 raft_consensus.cc:3060] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.739367  8107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f90b970828c470d9efd8522dee3ddde" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:11.740324  8107 raft_consensus.cc:2468] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 0.
I20260709 12:28:11.742240  8305 raft_consensus.cc:493] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.742764  8305 raft_consensus.cc:515] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.743170  8222 leader_election.cc:304] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.744886  8305 leader_election.cc:290] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.746935  8106 raft_consensus.cc:2468] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 1.
I20260709 12:28:11.747416  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abcb28adc6ec45029601c575e7640ae2" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:11.748078  8255 raft_consensus.cc:2468] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:11.748176  8182 raft_consensus.cc:2468] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 1.
I20260709 12:28:11.749346  8219 leader_election.cc:304] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.749720  8107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abcb28adc6ec45029601c575e7640ae2" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:11.749378  8146 leader_election.cc:304] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.750473  8107 raft_consensus.cc:2468] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:11.751015  8305 raft_consensus.cc:2804] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:11.751379  8305 raft_consensus.cc:493] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.751725  8305 raft_consensus.cc:3060] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.759498  8335 raft_consensus.cc:2804] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:11.760113  8335 raft_consensus.cc:493] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.760543  8335 raft_consensus.cc:3060] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.768283  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f90b970828c470d9efd8522dee3ddde" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59" is_pre_election: true
I20260709 12:28:11.769183  8182 raft_consensus.cc:2468] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 0.
I20260709 12:28:11.776409  8298 ts_tablet_manager.cc:1434] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.107s	user 0.048s	sys 0.042s
I20260709 12:28:11.774155  8305 raft_consensus.cc:515] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.774356  8303 leader_election.cc:290] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:11.775197  8332 raft_consensus.cc:2804] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:11.777957  8297 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:11.778330  8332 raft_consensus.cc:697] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:11.778537  8297 ts_tablet_manager.cc:1403] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.103s	user 0.054s	sys 0.027s
I20260709 12:28:11.779239  8332 consensus_queue.cc:237] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.779815  8107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abcb28adc6ec45029601c575e7640ae2" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:11.780449  8107 raft_consensus.cc:3060] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.781126  8305 leader_election.cc:290] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.780011  8296 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:11.781944  8296 ts_tablet_manager.cc:1403] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.105s	user 0.030s	sys 0.050s
I20260709 12:28:11.781795  8297 raft_consensus.cc:359] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.782610  8297 raft_consensus.cc:385] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.782917  8297 raft_consensus.cc:740] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.783756  8297 consensus_queue.cc:260] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.785341  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abcb28adc6ec45029601c575e7640ae2" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:11.785933  8255 raft_consensus.cc:3060] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.786635  8297 ts_tablet_manager.cc:1434] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:11.789569  8335 raft_consensus.cc:515] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.790352  8297 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.793656  8335 leader_election.cc:290] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:11.791384  8296 raft_consensus.cc:359] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.794265  8296 raft_consensus.cc:385] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.794591  8296 raft_consensus.cc:740] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.795107  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f90b970828c470d9efd8522dee3ddde" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:11.795755  8106 raft_consensus.cc:3060] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.795488  8296 consensus_queue.cc:260] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.797379  8107 raft_consensus.cc:2468] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:11.798177  8296 ts_tablet_manager.cc:1434] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 12:28:11.798823  8301 raft_consensus.cc:493] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.800870  8301 raft_consensus.cc:515] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.801082  8148 leader_election.cc:304] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:11.802639  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f90b970828c470d9efd8522dee3ddde" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:11.803263  8182 raft_consensus.cc:3060] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.802604  8305 raft_consensus.cc:2804] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:11.804325  8305 raft_consensus.cc:697] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:11.805178  8305 consensus_queue.cc:237] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.806345  8301 leader_election.cc:290] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.799466  8255 raft_consensus.cc:2468] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:11.813609  8296 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:11.816684  8003 catalog_manager.cc:5697] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 0 to 1, leader changed from <none> to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 1 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:11.814831  8298 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.820353  8182 raft_consensus.cc:2468] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 1.
I20260709 12:28:11.821640  8315 raft_consensus.cc:493] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.822175  8315 raft_consensus.cc:515] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.822944  8106 raft_consensus.cc:2468] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 1.
I20260709 12:28:11.823514  8219 leader_election.cc:304] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.824292  8315 leader_election.cc:290] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.826381  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:11.827054  8255 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:11.828176  8146 leader_election.cc:304] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.829046  8315 raft_consensus.cc:2804] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:11.829120  8332 raft_consensus.cc:2804] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:11.829512  8315 raft_consensus.cc:493] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.829720  8332 raft_consensus.cc:697] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:11.829907  8315 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.830758  8332 consensus_queue.cc:237] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:11.834794  8298 tablet_bootstrap.cc:654] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.840587  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:11.841301  8106 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:11.838084  8297 tablet_bootstrap.cc:654] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.837803  8315 raft_consensus.cc:515] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.838814  8296 tablet_bootstrap.cc:654] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.851122  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:11.851863  8106 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.864550  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f751c6cc07d44d28d60a92e61400400" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
W20260709 12:28:11.865926  8071 leader_election.cc:343] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:11.865957  8106 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:11.862147  8003 catalog_manager.cc:5697] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:11.866430  8005 catalog_manager.cc:5697] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 0 to 1, leader changed from <none> to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 1 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:11.867870  8148 leader_election.cc:304] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:11.868712  8305 raft_consensus.cc:2804] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:11.869174  8305 raft_consensus.cc:697] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:11.870103  8305 consensus_queue.cc:237] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.870792  8297 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:11.871284  8297 ts_tablet_manager.cc:1403] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.081s	user 0.012s	sys 0.005s
I20260709 12:28:11.869261  8298 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:11.872357  8315 leader_election.cc:290] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.872200  8298 ts_tablet_manager.cc:1403] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.012s
I20260709 12:28:11.874110  8297 raft_consensus.cc:359] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.877035  8297 raft_consensus.cc:385] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.877334  8297 raft_consensus.cc:740] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.878151  8297 consensus_queue.cc:260] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.877310  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:11.883183  8298 raft_consensus.cc:359] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.884030  8298 raft_consensus.cc:385] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.884402  8298 raft_consensus.cc:740] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.884685  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f751c6cc07d44d28d60a92e61400400" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59" is_pre_election: true
I20260709 12:28:11.885171  8298 consensus_queue.cc:260] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.884030  8255 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:11.886806  8070 leader_election.cc:343] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:11.887499  8070 leader_election.cc:304] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67; no voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:11.888191  8298 ts_tablet_manager.cc:1434] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.007s
I20260709 12:28:11.889317  8298 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.891571  8297 ts_tablet_manager.cc:1434] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:28:11.899483  8298 tablet_bootstrap.cc:654] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.901057  8301 raft_consensus.cc:493] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.901575  8301 raft_consensus.cc:515] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.906802  8255 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:11.909111  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:11.907622  8301 leader_election.cc:290] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.909837  8256 raft_consensus.cc:2468] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20dfbd3e445d44b48fec9c3dd8c42a67 in term 0.
I20260709 12:28:11.913164  8071 leader_election.cc:304] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:11.914080  8345 raft_consensus.cc:2804] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:11.914554  8345 raft_consensus.cc:493] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.914929  8345 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.920835  8301 raft_consensus.cc:2749] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:11.925375  8296 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:11.925931  8296 ts_tablet_manager.cc:1403] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.113s	user 0.017s	sys 0.004s
I20260709 12:28:11.929039  8003 catalog_manager.cc:5697] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:11.930541  8345 raft_consensus.cc:515] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.929082  8296 raft_consensus.cc:359] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.937846  8296 raft_consensus.cc:385] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.938237  8296 raft_consensus.cc:740] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.945201  8297 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:11.943634  8296 consensus_queue.cc:260] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.946306  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:11.947190  8256 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:11.949286  8296 ts_tablet_manager.cc:1434] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:28:11.950738  8296 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
W20260709 12:28:11.957036  8070 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:37049, user_credentials={real_user=slave}} blocked reactor thread for 70294us
I20260709 12:28:11.959569  8182 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59" is_pre_election: true
I20260709 12:28:11.961985  8296 tablet_bootstrap.cc:654] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.961812  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
W20260709 12:28:11.963141  8070 leader_election.cc:343] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:11.964318  8070 leader_election.cc:343] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:11.966346  8345 leader_election.cc:290] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 election: Requested vote from peers 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:11.969022  8256 raft_consensus.cc:2468] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20dfbd3e445d44b48fec9c3dd8c42a67 in term 1.
I20260709 12:28:11.970366  8298 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:11.970455  8071 leader_election.cc:304] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 712be1df6e174cd9928a200ac0a17e59
I20260709 12:28:11.971207  8298 ts_tablet_manager.cc:1403] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.082s	user 0.011s	sys 0.037s
I20260709 12:28:11.971966  8345 raft_consensus.cc:2804] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:11.972507  8297 tablet_bootstrap.cc:654] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.972508  8345 raft_consensus.cc:697] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 LEADER]: Becoming Leader. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Running, Role: LEADER
I20260709 12:28:11.973987  8296 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:11.973634  8345 consensus_queue.cc:237] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.974509  8296 ts_tablet_manager.cc:1403] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.007s
I20260709 12:28:11.975011  8298 raft_consensus.cc:359] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.975733  8298 raft_consensus.cc:385] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.976063  8298 raft_consensus.cc:740] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.976800  8298 consensus_queue.cc:260] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.977939  8296 raft_consensus.cc:359] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.978904  8296 raft_consensus.cc:385] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:11.979236  8296 raft_consensus.cc:740] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:11.980412  8296 consensus_queue.cc:260] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:11.982509  8298 ts_tablet_manager.cc:1434] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:28:11.983527  8298 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:11.989929  8298 tablet_bootstrap.cc:654] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:11.990449  8296 ts_tablet_manager.cc:1434] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 12:28:11.991498  8296 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:11.994607  8003 catalog_manager.cc:5697] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:11.998728  8296 tablet_bootstrap.cc:654] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:12.006035  8305 raft_consensus.cc:493] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.006789  8305 raft_consensus.cc:515] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.011233  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f751c6cc07d44d28d60a92e61400400" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:12.012080  8256 raft_consensus.cc:2468] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:12.014256  8146 leader_election.cc:304] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:12.014832  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f751c6cc07d44d28d60a92e61400400" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:12.015393  8315 raft_consensus.cc:2804] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:12.015727  8106 raft_consensus.cc:2468] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 0.
I20260709 12:28:12.016556  8305 leader_election.cc:290] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:12.015965  8315 raft_consensus.cc:493] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.023262  8315 raft_consensus.cc:3060] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.034749  8315 raft_consensus.cc:515] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.042047  8106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f751c6cc07d44d28d60a92e61400400" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:12.042866  8106 raft_consensus.cc:3060] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.049439  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f751c6cc07d44d28d60a92e61400400" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:12.049938  8315 leader_election.cc:290] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:12.050266  8256 raft_consensus.cc:3060] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.052991  8106 raft_consensus.cc:2468] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:12.053503  8296 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:12.053715  8298 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:12.053046  8297 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:12.054133  8296 ts_tablet_manager.cc:1403] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.063s	user 0.031s	sys 0.029s
I20260709 12:28:12.054411  8148 leader_election.cc:304] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:12.054548  8297 ts_tablet_manager.cc:1403] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.110s	user 0.047s	sys 0.039s
I20260709 12:28:12.054593  8298 ts_tablet_manager.cc:1403] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.021s
I20260709 12:28:12.057128  8296 raft_consensus.cc:359] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.057461  8297 raft_consensus.cc:359] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.057832  8296 raft_consensus.cc:385] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:12.058177  8297 raft_consensus.cc:385] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:12.057683  8298 raft_consensus.cc:359] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.058355  8296 raft_consensus.cc:740] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:12.058549  8297 raft_consensus.cc:740] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:12.058813  8298 raft_consensus.cc:385] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:12.059473  8298 raft_consensus.cc:740] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:12.059265  8296 consensus_queue.cc:260] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.058928  8256 raft_consensus.cc:2468] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:12.060791  8298 consensus_queue.cc:260] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.062150  8296 ts_tablet_manager.cc:1434] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:12.059336  8297 consensus_queue.cc:260] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.064491  8297 ts_tablet_manager.cc:1434] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:28:12.065438  8297 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:12.059079  8315 raft_consensus.cc:2804] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:12.062951  8298 ts_tablet_manager.cc:1434] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:28:12.068100  8298 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e: Bootstrap starting.
I20260709 12:28:12.069664  8296 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59: Bootstrap starting.
I20260709 12:28:12.071249  8297 tablet_bootstrap.cc:654] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:12.073534  8298 tablet_bootstrap.cc:654] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:12.076257  8296 tablet_bootstrap.cc:654] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:12.091656  8305 consensus_queue.cc:1048] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:12.094535  8315 raft_consensus.cc:697] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:12.096865  8305 consensus_queue.cc:1048] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:12.095484  8315 consensus_queue.cc:237] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.111063  8297 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: No bootstrap required, opened a new log
I20260709 12:28:12.111554  8297 ts_tablet_manager.cc:1403] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.016s
I20260709 12:28:12.114527  8297 raft_consensus.cc:359] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.115190  8297 raft_consensus.cc:385] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:12.115473  8297 raft_consensus.cc:740] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:12.115955  8298 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e: No bootstrap required, opened a new log
I20260709 12:28:12.116485  8298 ts_tablet_manager.cc:1403] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.019s
I20260709 12:28:12.116132  8297 consensus_queue.cc:260] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.118312  8297 ts_tablet_manager.cc:1434] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:28:12.126042  8298 raft_consensus.cc:359] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.126822  8298 raft_consensus.cc:385] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:12.127110  8298 raft_consensus.cc:740] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Initialized, Role: FOLLOWER
I20260709 12:28:12.131736  8298 consensus_queue.cc:260] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.133757  8298 ts_tablet_manager.cc:1434] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:28:12.138362  8318 consensus_queue.cc:1048] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:12.144469  8315 consensus_queue.cc:1048] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:12.146430  8296 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59: No bootstrap required, opened a new log
I20260709 12:28:12.146857  8296 ts_tablet_manager.cc:1403] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59: Time spent bootstrapping tablet: real 0.077s	user 0.023s	sys 0.003s
I20260709 12:28:12.147060  8345 raft_consensus.cc:493] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.147588  8345 raft_consensus.cc:515] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.149724  8345 leader_election.cc:290] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:12.149564  8296 raft_consensus.cc:359] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.151208  8296 raft_consensus.cc:385] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:12.151554  8296 raft_consensus.cc:740] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Initialized, Role: FOLLOWER
I20260709 12:28:12.151690  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:12.152432  8256 raft_consensus.cc:2468] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20dfbd3e445d44b48fec9c3dd8c42a67 in term 0.
I20260709 12:28:12.152632  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59" is_pre_election: true
I20260709 12:28:12.153968  8071 leader_election.cc:304] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:12.153282  8003 catalog_manager.cc:5697] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:12.155067  8345 raft_consensus.cc:2804] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:12.155452  8345 raft_consensus.cc:493] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.155918  8345 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.152320  8296 consensus_queue.cc:260] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.157089  8181 raft_consensus.cc:2468] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20dfbd3e445d44b48fec9c3dd8c42a67 in term 0.
I20260709 12:28:12.161410  8296 ts_tablet_manager.cc:1434] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:28:12.168051  8345 raft_consensus.cc:515] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.171912  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:12.172465  8181 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.173786  8345 leader_election.cc:290] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 election: Requested vote from peers 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:12.174062  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa" candidate_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:12.174708  8256 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.180022  8181 raft_consensus.cc:2468] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20dfbd3e445d44b48fec9c3dd8c42a67 in term 1.
I20260709 12:28:12.181043  8070 leader_election.cc:304] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:12.181999  8346 raft_consensus.cc:2804] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:12.182193  8256 raft_consensus.cc:2468] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20dfbd3e445d44b48fec9c3dd8c42a67 in term 1.
I20260709 12:28:12.182602  8346 raft_consensus.cc:697] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 LEADER]: Becoming Leader. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Running, Role: LEADER
I20260709 12:28:12.183552  8346 consensus_queue.cc:237] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.192480  8003 catalog_manager.cc:5697] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:12.217577  8332 raft_consensus.cc:493] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.218071  8332 raft_consensus.cc:515] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.219717  8332 leader_election.cc:290] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:12.220481  8107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bb19f5d45240ef8d5b1facd4c12e45" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:12.220723  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bb19f5d45240ef8d5b1facd4c12e45" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59" is_pre_election: true
I20260709 12:28:12.221088  8107 raft_consensus.cc:2468] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 0.
I20260709 12:28:12.221328  8181 raft_consensus.cc:2468] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 0.
I20260709 12:28:12.222193  8222 leader_election.cc:304] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:12.222906  8332 raft_consensus.cc:2804] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:12.223307  8332 raft_consensus.cc:493] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.223578  8332 raft_consensus.cc:3060] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.228647  8332 raft_consensus.cc:515] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.230177  8332 leader_election.cc:290] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:12.231117  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bb19f5d45240ef8d5b1facd4c12e45" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:12.231027  8107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bb19f5d45240ef8d5b1facd4c12e45" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:12.231734  8181 raft_consensus.cc:3060] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.231791  8107 raft_consensus.cc:3060] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.238673  8107 raft_consensus.cc:2468] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 1.
I20260709 12:28:12.238673  8181 raft_consensus.cc:2468] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 1.
I20260709 12:28:12.239916  8219 leader_election.cc:304] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:12.240662  8332 raft_consensus.cc:2804] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:12.241164  8332 raft_consensus.cc:697] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:12.241992  8332 consensus_queue.cc:237] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.242980  8335 consensus_queue.cc:1048] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:12.249035  8363 raft_consensus.cc:493] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.249523  8363 raft_consensus.cc:515] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:12.251561  8363 leader_election.cc:290] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:12.250766  8003 catalog_manager.cc:5697] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 0 to 1, leader changed from <none> to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 1 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:12.253760  8256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:12.254575  8256 raft_consensus.cc:2393] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 712be1df6e174cd9928a200ac0a17e59 in current term 1: Already voted for candidate 20dfbd3e445d44b48fec9c3dd8c42a67 in this term.
I20260709 12:28:12.255697  8107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" is_pre_election: true
I20260709 12:28:12.256668  8335 consensus_queue.cc:1048] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:12.257113  8148 leader_election.cc:304] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59; no voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:12.258448  8363 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.265731  8363 raft_consensus.cc:2749] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:12.269367  8363 consensus_queue.cc:1048] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:12.285231  8364 consensus_queue.cc:1048] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:12.327005 29845 ts_manager.cc:295] Set tserver state for 20dfbd3e445d44b48fec9c3dd8c42a67 to MAINTENANCE_MODE
I20260709 12:28:12.327693 29845 tablet_server.cc:179] TabletServer@127.29.37.65:0 shutting down...
I20260709 12:28:12.347488 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:12.348388 29845 tablet_replica.cc:333] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
W20260709 12:28:12.348635  8148 consensus_peers.cc:597] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:12.348978 29845 raft_consensus.cc:2243] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.349550 29845 raft_consensus.cc:2272] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.351707 29845 tablet_replica.cc:333] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.352260 29845 raft_consensus.cc:2243] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.352960 29845 pending_rounds.cc:70] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Trying to abort 1 pending ops.
I20260709 12:28:12.353176 29845 pending_rounds.cc:77] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305625976734543872 op_type: NO_OP noop_request { }
I20260709 12:28:12.353523 29845 raft_consensus.cc:2889] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:12.353797 29845 raft_consensus.cc:2272] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.356029 29845 tablet_replica.cc:333] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.356573 29845 raft_consensus.cc:2243] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.356947 29845 raft_consensus.cc:2272] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.358633 29845 tablet_replica.cc:333] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.359205 29845 raft_consensus.cc:2243] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.359669 29845 raft_consensus.cc:2272] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.361389 29845 tablet_replica.cc:333] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.361977 29845 raft_consensus.cc:2243] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.362398 29845 raft_consensus.cc:2272] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.364382 29845 tablet_replica.cc:333] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.364919 29845 raft_consensus.cc:2243] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.365334 29845 raft_consensus.cc:2272] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.368131 29845 tablet_replica.cc:333] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.368057  8364 consensus_queue.cc:1048] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:12.368839 29845 raft_consensus.cc:2243] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.369575 29845 pending_rounds.cc:70] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Trying to abort 1 pending ops.
I20260709 12:28:12.369750 29845 pending_rounds.cc:77] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305625977594277888 op_type: NO_OP noop_request { }
I20260709 12:28:12.370075 29845 raft_consensus.cc:2889] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:12.370425 29845 raft_consensus.cc:2272] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.373867 29845 tablet_replica.cc:333] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.374586 29845 raft_consensus.cc:2243] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.375064 29845 raft_consensus.cc:2272] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.377938 29845 tablet_replica.cc:333] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.378690 29845 raft_consensus.cc:2243] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.379156 29845 raft_consensus.cc:2272] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.380955 29845 tablet_replica.cc:333] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: stopping tablet replica
I20260709 12:28:12.381505 29845 raft_consensus.cc:2243] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.381986 29845 raft_consensus.cc:2272] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.388711  8335 consensus_queue.cc:1048] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:12.404482  8222 consensus_peers.cc:597] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:12.424777 29845 tablet_server.cc:196] TabletServer@127.29.37.65:0 shutdown complete.
W20260709 12:28:12.521544  8148 consensus_peers.cc:597] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.577946  8148 consensus_peers.cc:597] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.624769  8222 consensus_peers.cc:597] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.655056  8148 consensus_peers.cc:597] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.677922  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:12.703104  8222 consensus_peers.cc:597] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:12.703930  8361 consensus_queue.cc:1048] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:12.807771  8335 consensus_queue.cc:1048] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:12.885437  8148 consensus_peers.cc:597] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:13.679023  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:13.986583  8361 raft_consensus.cc:493] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.987134  8361 raft_consensus.cc:515] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:13.988875  8361 leader_election.cc:290] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:13.989982  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
W20260709 12:28:13.990176  8148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111) [suppressed 117 similar messages]
I20260709 12:28:13.990739  8255 raft_consensus.cc:2468] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:13.991793  8146 leader_election.cc:304] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:13.992544  8361 raft_consensus.cc:2804] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:13.992908  8361 raft_consensus.cc:493] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:13.993060  8148 leader_election.cc:336] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111)
I20260709 12:28:13.993274  8361 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:13.998308  8361 raft_consensus.cc:515] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:13.999742  8361 leader_election.cc:290] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:14.000830  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:14.001435  8255 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:14.003333  8148 leader_election.cc:336] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111)
I20260709 12:28:14.007018  8255 raft_consensus.cc:2468] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 2.
I20260709 12:28:14.008083  8146 leader_election.cc:304] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 20dfbd3e445d44b48fec9c3dd8c42a67
I20260709 12:28:14.008881  8361 raft_consensus.cc:2804] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:14.009897  8361 raft_consensus.cc:697] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:14.010805  8361 consensus_queue.cc:237] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:14.017688  8003 catalog_manager.cc:5697] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 1 to 2, leader changed from 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65) to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:14.055506  8361 raft_consensus.cc:493] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.055944  8361 raft_consensus.cc:515] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:14.057750  8361 leader_election.cc:290] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:14.058903  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e" is_pre_election: true
I20260709 12:28:14.059648  8255 raft_consensus.cc:2468] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 1.
I20260709 12:28:14.060654  8146 leader_election.cc:304] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 
W20260709 12:28:14.061295  8148 leader_election.cc:336] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111)
I20260709 12:28:14.061436  8361 raft_consensus.cc:2804] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:14.061841  8361 raft_consensus.cc:493] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.062167  8361 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.066982  8361 raft_consensus.cc:515] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:14.068565  8361 leader_election.cc:290] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:14.069636  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:14.070267  8255 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:14.071918  8148 leader_election.cc:336] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111)
I20260709 12:28:14.075464  8255 raft_consensus.cc:2468] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 2.
I20260709 12:28:14.076400  8146 leader_election.cc:304] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 20dfbd3e445d44b48fec9c3dd8c42a67
I20260709 12:28:14.077041  8361 raft_consensus.cc:2804] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:14.077451  8361 raft_consensus.cc:697] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:14.078065  8361 consensus_queue.cc:237] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:14.084438  8003 catalog_manager.cc:5697] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 1 to 2, leader changed from 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65) to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:14.417583  8148 consensus_peers.cc:597] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:14.473055  8148 consensus_peers.cc:597] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:14.524415  8255 raft_consensus.cc:1275] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:28:14.525820  8361 consensus_queue.cc:1048] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:14.656579  8255 raft_consensus.cc:1275] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:28:14.657732  8361 consensus_queue.cc:1048] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:14.679827  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:14.740336  8148 consensus_peers.cc:597] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:14.815948  8148 consensus_peers.cc:597] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:14.910638  8222 consensus_peers.cc:597] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:14.922892  8148 consensus_peers.cc:597] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:15.109771  8222 consensus_peers.cc:597] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:15.119647  8148 consensus_peers.cc:597] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:15.192344  8222 consensus_peers.cc:597] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:15.250939  8148 consensus_peers.cc:597] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:15.681106  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:16.682602  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:16.790335  8148 consensus_peers.cc:597] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:17.110141  8148 consensus_peers.cc:597] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:17.156824  8148 consensus_peers.cc:597] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:17.174991  8148 consensus_peers.cc:597] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:17.240500  8148 consensus_peers.cc:597] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:17.445663 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:17.453172  8397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:17.454613  8398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:17.458084 29845 server_base.cc:1061] running on GCE node
W20260709 12:28:17.458581  8400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:17.462906 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:17.463155 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:17.463354 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600097463337 us; error 0 us; skew 500 ppm
I20260709 12:28:17.463954 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:17.466598 29845 webserver.cc:533] Webserver started at http://127.29.37.65:38169/ using document root <none> and password file <none>
I20260709 12:28:17.467157 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:17.467315 29845 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:28:17.471342 29845 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:28:17.475350  8405 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:17.476104 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:28:17.476404 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
format_stamp: "Formatted at 2026-07-09 12:28:10 on dist-test-slave-g1m6"
I20260709 12:28:17.476819 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:17.490662 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:17.491887 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:17.493705 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:17.498454  8412 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:28:17.545253 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:28:17.545591 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.048s	user 0.001s	sys 0.001s
I20260709 12:28:17.545898 29845 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:28:17.552012  8412 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.565512  8412 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:17.566309  8412 tablet_bootstrap.cc:492] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.566844  8412 ts_tablet_manager.cc:1403] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.015s	user 0.002s	sys 0.012s
I20260709 12:28:17.569105  8412 raft_consensus.cc:359] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:17.569911  8412 raft_consensus.cc:740] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.570600  8412 consensus_queue.cc:260] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:17.572865  8412 ts_tablet_manager.cc:1434] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.005s
I20260709 12:28:17.573545  8412 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.585515  8412 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:17.586380  8412 tablet_bootstrap.cc:492] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.586382 29845 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:28:17.586844 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.041s	user 0.039s	sys 0.002s
I20260709 12:28:17.587227  8412 ts_tablet_manager.cc:1403] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:28:17.589716  8412 raft_consensus.cc:359] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.590529  8412 raft_consensus.cc:740] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.591346  8412 consensus_queue.cc:260] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.593286  8412 ts_tablet_manager.cc:1434] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:17.594187  8412 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.609193  8412 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:17.610018  8412 tablet_bootstrap.cc:492] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.610764  8412 ts_tablet_manager.cc:1403] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 12:28:17.613907  8412 raft_consensus.cc:359] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.614645  8412 raft_consensus.cc:740] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.615546  8412 consensus_queue.cc:260] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.618083  8412 ts_tablet_manager.cc:1434] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:17.619293  8412 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.637400  8412 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:17.638499  8412 tablet_bootstrap.cc:492] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.639376  8412 ts_tablet_manager.cc:1403] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 12:28:17.642990  8412 raft_consensus.cc:359] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.643911  8412 raft_consensus.cc:740] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.644845  8412 consensus_queue.cc:260] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.647660  8412 ts_tablet_manager.cc:1434] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:17.648723  8412 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.649060 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:35385
I20260709 12:28:17.649202  8473 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:35385 every 8 connection(s)
W20260709 12:28:17.666391  8477 tablet.cc:2406] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:28:17.667596 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:17.683780  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:17.705601  8475 heartbeater.cc:344] Connected to a master server at 127.29.37.126:37501
I20260709 12:28:17.706121  8475 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:17.707080  8475 heartbeater.cc:507] Master 127.29.37.126:37501 requested a full tablet report, sending...
I20260709 12:28:17.708348  8412 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:17.709424  8412 tablet_bootstrap.cc:492] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.710166  8412 ts_tablet_manager.cc:1403] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.016s
I20260709 12:28:17.713390  8412 raft_consensus.cc:359] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.714356  8412 raft_consensus.cc:740] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.715144  8412 consensus_queue.cc:260] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.717404  8412 ts_tablet_manager.cc:1434] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:28:17.718394  8412 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.737216  8181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86e376d584f94d5c84247a21cb2b0d76"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
mode: GRACEFUL
new_leader_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:59926
I20260709 12:28:17.737944  8181 raft_consensus.cc:606] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Received request to transfer leadership to TS 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:17.738260  8005 ts_manager.cc:194] Re-registered known tserver with Master: 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385)
I20260709 12:28:17.739588  8412 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:28:17.740589  8412 tablet_bootstrap.cc:492] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.741199  8412 ts_tablet_manager.cc:1403] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 12:28:17.741019  8181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8da354781025485892cf19d483eb32a3"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
mode: GRACEFUL
new_leader_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:59926
I20260709 12:28:17.741739  8181 raft_consensus.cc:606] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Received request to transfer leadership to TS 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:17.744022  8412 raft_consensus.cc:359] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.744983  8412 raft_consensus.cc:740] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.744872  8181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abcb28adc6ec45029601c575e7640ae2"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
mode: GRACEFUL
new_leader_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:59926
I20260709 12:28:17.745684  8181 raft_consensus.cc:606] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Received request to transfer leadership to TS 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:17.745886  8412 consensus_queue.cc:260] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.747043 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:28:17.747716 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:17.748514 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:17.749094  8412 ts_tablet_manager.cc:1434] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:28:17.749717  8005 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35120
I20260709 12:28:17.750087  8412 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.753839  8475 heartbeater.cc:499] Master 127.29.37.126:37501 was elected leader, sending a full tablet report...
I20260709 12:28:17.763873  8412 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:17.764820  8412 tablet_bootstrap.cc:492] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.765434  8412 ts_tablet_manager.cc:1403] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:28:17.767964  8412 raft_consensus.cc:359] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.768682  8412 raft_consensus.cc:740] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.769429  8412 consensus_queue.cc:260] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.771410  8412 ts_tablet_manager.cc:1434] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:28:17.772033  8412 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.777230  8447 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.785544  8490 raft_consensus.cc:993] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59: : Instructing follower 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:17.786324  8412 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:17.786265  8395 raft_consensus.cc:1081] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Signalling peer 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:17.787235  8412 tablet_bootstrap.cc:492] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.787813  8412 ts_tablet_manager.cc:1403] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.003s
I20260709 12:28:17.788376  8255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86e376d584f94d5c84247a21cb2b0d76"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:45978
I20260709 12:28:17.789021  8255 raft_consensus.cc:493] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:17.789170  8447 pending_rounds.cc:85] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Aborting all ops after (but not including) 0
I20260709 12:28:17.789481  8255 raft_consensus.cc:3060] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.789589  8447 pending_rounds.cc:107] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:28:17.790072  8447 raft_consensus.cc:2889] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:17.790524  8412 raft_consensus.cc:359] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.791332  8412 raft_consensus.cc:740] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.792182  8412 consensus_queue.cc:260] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.794468  8412 ts_tablet_manager.cc:1434] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:28:17.795324  8412 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.798029  8255 raft_consensus.cc:515] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.801321  8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e376d584f94d5c84247a21cb2b0d76" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
W20260709 12:28:17.802977  8222 leader_election.cc:343] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:17.804747  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86e376d584f94d5c84247a21cb2b0d76" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:17.805408  8181 raft_consensus.cc:3055] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:17.805770  8181 raft_consensus.cc:740] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:17.802990  8255 leader_election.cc:290] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:17.808060  8181 consensus_queue.cc:260] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.809587  8181 raft_consensus.cc:3060] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.813632  8412 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:17.814606  8412 tablet_bootstrap.cc:492] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.815166  8412 ts_tablet_manager.cc:1403] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 12:28:17.817044  8412 raft_consensus.cc:359] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.817528  8412 raft_consensus.cc:740] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.817446  8181 raft_consensus.cc:2468] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 2.
I20260709 12:28:17.818429  8412 consensus_queue.cc:260] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.819312  8219 leader_election.cc:304] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 712be1df6e174cd9928a200ac0a17e59, 72fb489f0a054602bb946316bc7c187e; no voters: 20dfbd3e445d44b48fec9c3dd8c42a67
I20260709 12:28:17.820271  8335 raft_consensus.cc:2804] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:17.820470  8412 ts_tablet_manager.cc:1434] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:17.821367  8412 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap starting.
I20260709 12:28:17.821669  8335 raft_consensus.cc:697] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:17.822692  8335 consensus_queue.cc:237] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.833052  8003 catalog_manager.cc:5697] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 1 to 2, leader changed from 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66) to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 2 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.835134  8412 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:28:17.835875  8412 tablet_bootstrap.cc:492] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Bootstrap complete.
I20260709 12:28:17.836441  8412 ts_tablet_manager.cc:1403] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent bootstrapping tablet: real 0.015s	user 0.015s	sys 0.000s
I20260709 12:28:17.838518  8412 raft_consensus.cc:359] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.839119  8412 raft_consensus.cc:740] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20dfbd3e445d44b48fec9c3dd8c42a67, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.839792  8412 consensus_queue.cc:260] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.841437  8412 ts_tablet_manager.cc:1434] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 12:28:17.948465  8395 raft_consensus.cc:993] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59: : Instructing follower 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:17.948963  8395 raft_consensus.cc:1081] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Signalling peer 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:17.950522  8255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8da354781025485892cf19d483eb32a3"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:45978
I20260709 12:28:17.951022  8255 raft_consensus.cc:493] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:17.951332  8255 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.956764  8255 raft_consensus.cc:515] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.958294  8255 leader_election.cc:290] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:17.959134  8447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:17.959460  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:17.959842  8447 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.960139  8181 raft_consensus.cc:3055] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:17.960546  8181 raft_consensus.cc:740] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:17.961508  8181 consensus_queue.cc:260] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.962560  8181 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.965529  8447 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 2.
I20260709 12:28:17.966207  8446 raft_consensus.cc:1240] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Rejecting Update request from peer 712be1df6e174cd9928a200ac0a17e59 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:17.966651  8222 leader_election.cc:304] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:17.967705  8335 raft_consensus.cc:2804] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:17.967988  8181 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 2.
I20260709 12:28:17.968266  8335 raft_consensus.cc:697] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:17.969362  8335 consensus_queue.cc:237] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:17.977459  8003 catalog_manager.cc:5697] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 1 to 2, leader changed from 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66) to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 2 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:18.023816  8490 raft_consensus.cc:993] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59: : Instructing follower 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:18.024308  8395 raft_consensus.cc:1081] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Signalling peer 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:18.025645  8255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abcb28adc6ec45029601c575e7640ae2"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:45978
I20260709 12:28:18.026145  8255 raft_consensus.cc:493] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:18.026422  8255 raft_consensus.cc:3060] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:18.032160  8255 raft_consensus.cc:515] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.033533  8255 leader_election.cc:290] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:18.034473  8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abcb28adc6ec45029601c575e7640ae2" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:18.034485  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abcb28adc6ec45029601c575e7640ae2" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:18.035127  8446 raft_consensus.cc:3060] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:18.035228  8181 raft_consensus.cc:3055] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:18.035621  8181 raft_consensus.cc:740] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:18.036295  8181 consensus_queue.cc:260] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.037298  8181 raft_consensus.cc:3060] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:18.040745  8446 raft_consensus.cc:2468] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 2.
I20260709 12:28:18.041906  8222 leader_election.cc:304] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:18.042307  8181 raft_consensus.cc:2468] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 2.
I20260709 12:28:18.042655  8335 raft_consensus.cc:2804] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:18.043184  8335 raft_consensus.cc:697] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:18.044044  8335 consensus_queue.cc:237] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.050537  8003 catalog_manager.cc:5697] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 1 to 2, leader changed from 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66) to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 2 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:18.219405  8446 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:18.224546  8446 pending_rounds.cc:85] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Aborting all ops after (but not including) 0
I20260709 12:28:18.224785  8446 pending_rounds.cc:107] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:28:18.225112  8446 raft_consensus.cc:2889] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:18.225605  8181 raft_consensus.cc:1275] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.227427  8504 consensus_queue.cc:1048] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:18.242527  8446 raft_consensus.cc:3060] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:18.249185  8446 raft_consensus.cc:1275] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.250574  8495 consensus_queue.cc:1048] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.377231  8446 raft_consensus.cc:1275] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.378501  8495 consensus_queue.cc:1048] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.386998  8181 raft_consensus.cc:1275] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.389087  8495 consensus_queue.cc:1048] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.482052  8446 raft_consensus.cc:1275] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.483363  8495 consensus_queue.cc:1048] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.490707  8181 raft_consensus.cc:1275] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.492339  8495 consensus_queue.cc:1048] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:18.685009  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:18.758669 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:18.777779  8255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17bb19f5d45240ef8d5b1facd4c12e45"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
mode: GRACEFUL
new_leader_uuid: "712be1df6e174cd9928a200ac0a17e59"
 from {username='slave'} at 127.0.0.1:55206
I20260709 12:28:18.778258  8255 raft_consensus.cc:606] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Received request to transfer leadership to TS 712be1df6e174cd9928a200ac0a17e59
I20260709 12:28:18.780604  8255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8da354781025485892cf19d483eb32a3"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
mode: GRACEFUL
new_leader_uuid: "712be1df6e174cd9928a200ac0a17e59"
 from {username='slave'} at 127.0.0.1:55206
I20260709 12:28:18.781024  8255 raft_consensus.cc:606] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Received request to transfer leadership to TS 712be1df6e174cd9928a200ac0a17e59
I20260709 12:28:18.781953 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:18.782441 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:18.782932 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:18.847530  8335 raft_consensus.cc:993] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e: : Instructing follower 712be1df6e174cd9928a200ac0a17e59 to start an election
I20260709 12:28:18.848014  8495 raft_consensus.cc:1081] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Signalling peer 712be1df6e174cd9928a200ac0a17e59 to start an election
I20260709 12:28:18.849421  8181 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17bb19f5d45240ef8d5b1facd4c12e45"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
 from {username='slave'} at 127.0.0.1:59902
I20260709 12:28:18.849852  8181 raft_consensus.cc:493] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:18.850101  8181 raft_consensus.cc:3060] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:18.855479  8181 raft_consensus.cc:515] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.857107  8181 leader_election.cc:290] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:18.857818  8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bb19f5d45240ef8d5b1facd4c12e45" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:18.858282  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17bb19f5d45240ef8d5b1facd4c12e45" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:18.858490  8446 raft_consensus.cc:3060] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:18.858901  8255 raft_consensus.cc:3055] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:18.859287  8255 raft_consensus.cc:740] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:18.860059  8255 consensus_queue.cc:260] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.861441  8255 raft_consensus.cc:3060] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:18.864365  8446 raft_consensus.cc:2468] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 2.
I20260709 12:28:18.865530  8148 leader_election.cc:304] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:18.866314  8490 raft_consensus.cc:2804] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:18.866349  8255 raft_consensus.cc:2468] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 2.
I20260709 12:28:18.866755  8490 raft_consensus.cc:697] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:18.867795  8490 consensus_queue.cc:237] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.874665  8003 catalog_manager.cc:5697] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 1 to 2, leader changed from 72fb489f0a054602bb946316bc7c187e (127.29.37.67) to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:18.973942  8495 raft_consensus.cc:993] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e: : Instructing follower 712be1df6e174cd9928a200ac0a17e59 to start an election
I20260709 12:28:18.974478  8512 raft_consensus.cc:1081] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Signalling peer 712be1df6e174cd9928a200ac0a17e59 to start an election
I20260709 12:28:18.975953  8181 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8da354781025485892cf19d483eb32a3"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
 from {username='slave'} at 127.0.0.1:59902
I20260709 12:28:18.976493  8181 raft_consensus.cc:493] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:18.976770  8181 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:18.981647  8181 raft_consensus.cc:515] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.983314  8181 leader_election.cc:290] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 3 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:18.983997  8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:18.984354  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:18.984620  8446 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:18.984905  8255 raft_consensus.cc:3055] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:18.985212  8255 raft_consensus.cc:740] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:18.985852  8255 consensus_queue.cc:260] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.986716  8255 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:18.990768  8446 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 3.
I20260709 12:28:18.991770  8148 leader_election.cc:304] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:18.992506  8490 raft_consensus.cc:2804] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:18.992923  8255 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 3.
I20260709 12:28:18.993079  8490 raft_consensus.cc:697] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 3 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:18.994059  8490 consensus_queue.cc:237] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:18.999729  8003 catalog_manager.cc:5697] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 2 to 3, leader changed from 72fb489f0a054602bb946316bc7c187e (127.29.37.67) to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 3 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:19.300705  8255 raft_consensus.cc:1275] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:19.301949  8490 consensus_queue.cc:1048] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:19.312879  8447 raft_consensus.cc:1275] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:19.314826  8395 consensus_queue.cc:1048] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:19.396576  8255 raft_consensus.cc:1275] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:19.397769  8523 consensus_queue.cc:1048] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:19.406006  8447 raft_consensus.cc:1275] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:19.407564  8490 consensus_queue.cc:1048] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:28:19.686386  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:19.789446 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:19.797338  8181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
mode: GRACEFUL
new_leader_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:59926
I20260709 12:28:19.797825  8181 raft_consensus.cc:606] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Received request to transfer leadership to TS 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:19.800168  8181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
mode: GRACEFUL
new_leader_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:59926
I20260709 12:28:19.800592  8181 raft_consensus.cc:606] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Received request to transfer leadership to TS 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:19.801577 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:19.801994 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:19.802536 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:19.925119  8523 raft_consensus.cc:993] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59: : Instructing follower 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:19.925690  8490 raft_consensus.cc:1081] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Signalling peer 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:19.927168  8255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:45978
I20260709 12:28:19.927613  8255 raft_consensus.cc:493] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:19.927882  8255 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:19.932792  8255 raft_consensus.cc:515] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:19.934293  8255 leader_election.cc:290] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 3 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:19.935129  8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:19.935376  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27a7c938363946bf9efcf432ea6d935a" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:19.935815  8446 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:19.935962  8181 raft_consensus.cc:3055] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:19.936329  8181 raft_consensus.cc:740] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:19.936991  8181 consensus_queue.cc:260] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:19.937896  8181 raft_consensus.cc:3060] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:19.941114  8446 raft_consensus.cc:2468] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 3.
I20260709 12:28:19.942399  8181 raft_consensus.cc:2468] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 3.
I20260709 12:28:19.942337  8222 leader_election.cc:304] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:19.943352  8512 raft_consensus.cc:2804] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:19.944191  8512 raft_consensus.cc:697] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 3 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:19.945135  8512 consensus_queue.cc:237] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:19.951500  8005 catalog_manager.cc:5697] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 2 to 3, leader changed from 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66) to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 3 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:20.145608  8523 raft_consensus.cc:993] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59: : Instructing follower 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:20.146073  8523 raft_consensus.cc:1081] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Signalling peer 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:20.147462  8255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:45978
I20260709 12:28:20.147907  8255 raft_consensus.cc:493] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:20.148187  8255 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:20.152771  8255 raft_consensus.cc:515] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:20.154189  8255 leader_election.cc:290] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 3 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:20.155052  8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:20.155277  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2face0b298ca4aa7a64cc288d6c5f6fa" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:20.155697  8446 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:20.155789  8181 raft_consensus.cc:3055] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:20.156173  8181 raft_consensus.cc:740] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:20.156922  8181 consensus_queue.cc:260] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:20.158152  8181 raft_consensus.cc:3060] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:20.162530  8446 raft_consensus.cc:2468] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 3.
I20260709 12:28:20.163651  8222 leader_election.cc:304] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:20.164407  8512 raft_consensus.cc:2804] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:20.164845  8512 raft_consensus.cc:697] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 3 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:20.164955  8181 raft_consensus.cc:2468] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 3.
I20260709 12:28:20.165783  8512 consensus_queue.cc:237] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:20.172966  8005 catalog_manager.cc:5697] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 2 to 3, leader changed from 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66) to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 3 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:20.397329  8446 raft_consensus.cc:1275] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:20.398622  8512 consensus_queue.cc:1048] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:20.410615  8181 raft_consensus.cc:1275] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:20.413024  8530 consensus_queue.cc:1048] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:20.556368  8181 raft_consensus.cc:1275] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:20.557717  8530 consensus_queue.cc:1048] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:20.567034  8447 raft_consensus.cc:1275] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:20.568994  8512 consensus_queue.cc:1048] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:20.687489  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:20.808454 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:20.817871  8255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f90b970828c470d9efd8522dee3ddde"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
mode: GRACEFUL
new_leader_uuid: "712be1df6e174cd9928a200ac0a17e59"
 from {username='slave'} at 127.0.0.1:55206
I20260709 12:28:20.818435  8255 raft_consensus.cc:606] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Received request to transfer leadership to TS 712be1df6e174cd9928a200ac0a17e59
I20260709 12:28:20.820660  8255 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4d449e67e41460db81d4df8bff90f65"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
mode: GRACEFUL
new_leader_uuid: "712be1df6e174cd9928a200ac0a17e59"
 from {username='slave'} at 127.0.0.1:55206
I20260709 12:28:20.821085  8255 raft_consensus.cc:606] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Received request to transfer leadership to TS 712be1df6e174cd9928a200ac0a17e59
I20260709 12:28:20.821992 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:20.822510 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:20.823119 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:21.178907  8512 raft_consensus.cc:993] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e: : Instructing follower 712be1df6e174cd9928a200ac0a17e59 to start an election
I20260709 12:28:21.179388  8544 raft_consensus.cc:1081] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Signalling peer 712be1df6e174cd9928a200ac0a17e59 to start an election
I20260709 12:28:21.181027  8181 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f90b970828c470d9efd8522dee3ddde"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
 from {username='slave'} at 127.0.0.1:59902
I20260709 12:28:21.181425  8181 raft_consensus.cc:493] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:21.181644  8181 raft_consensus.cc:3060] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:21.186292  8181 raft_consensus.cc:515] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:21.187826  8181 leader_election.cc:290] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:21.188572  8447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f90b970828c470d9efd8522dee3ddde" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:21.188884  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f90b970828c470d9efd8522dee3ddde" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:21.189234  8447 raft_consensus.cc:3060] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:21.189482  8255 raft_consensus.cc:3055] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:21.189857  8255 raft_consensus.cc:740] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:21.190642  8255 consensus_queue.cc:260] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:21.191581  8255 raft_consensus.cc:3060] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:21.194454  8447 raft_consensus.cc:2468] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 2.
I20260709 12:28:21.195531  8148 leader_election.cc:304] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:21.196187  8523 raft_consensus.cc:2804] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:21.196503  8255 raft_consensus.cc:2468] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 2.
I20260709 12:28:21.196691  8523 raft_consensus.cc:697] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:21.197777  8523 consensus_queue.cc:237] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } }
I20260709 12:28:21.205034  8005 catalog_manager.cc:5697] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 1 to 2, leader changed from 72fb489f0a054602bb946316bc7c187e (127.29.37.67) to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.380924  8512 raft_consensus.cc:993] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e: : Instructing follower 712be1df6e174cd9928a200ac0a17e59 to start an election
I20260709 12:28:21.381408  8544 raft_consensus.cc:1081] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Signalling peer 712be1df6e174cd9928a200ac0a17e59 to start an election
I20260709 12:28:21.382841  8181 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4d449e67e41460db81d4df8bff90f65"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
 from {username='slave'} at 127.0.0.1:59902
I20260709 12:28:21.383230  8181 raft_consensus.cc:493] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:21.383458  8181 raft_consensus.cc:3060] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:21.388432  8181 raft_consensus.cc:515] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:21.390240  8181 leader_election.cc:290] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 72fb489f0a054602bb946316bc7c187e (127.29.37.67:35273)
I20260709 12:28:21.391275  8255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d449e67e41460db81d4df8bff90f65" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "72fb489f0a054602bb946316bc7c187e"
I20260709 12:28:21.391171  8447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d449e67e41460db81d4df8bff90f65" candidate_uuid: "712be1df6e174cd9928a200ac0a17e59" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:21.391985  8255 raft_consensus.cc:3055] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:21.392051  8447 raft_consensus.cc:3060] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:21.392382  8255 raft_consensus.cc:740] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:21.393252  8255 consensus_queue.cc:260] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:21.394515  8255 raft_consensus.cc:3060] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:21.399114  8447 raft_consensus.cc:2468] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 2.
I20260709 12:28:21.400307  8148 leader_election.cc:304] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 712be1df6e174cd9928a200ac0a17e59; no voters: 
I20260709 12:28:21.401072  8523 raft_consensus.cc:2804] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:21.401268  8255 raft_consensus.cc:2468] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712be1df6e174cd9928a200ac0a17e59 in term 2.
I20260709 12:28:21.401489  8523 raft_consensus.cc:697] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Becoming Leader. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:21.402537  8523 consensus_queue.cc:237] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:21.409238  8005 catalog_manager.cc:5697] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 reported cstate change: term changed from 1 to 2, leader changed from 72fb489f0a054602bb946316bc7c187e (127.29.37.67) to 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "712be1df6e174cd9928a200ac0a17e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.586516  8255 raft_consensus.cc:1275] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:21.587925  8523 consensus_queue.cc:1048] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:21.588331  8447 raft_consensus.cc:1275] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:21.589653  8523 consensus_queue.cc:1048] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:21.688406  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:21.822516  8255 raft_consensus.cc:1275] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:21.823832  8523 consensus_queue.cc:1048] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:21.829820 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:21.832216  8446 raft_consensus.cc:1275] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Refusing update from remote peer 712be1df6e174cd9928a200ac0a17e59: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:21.834095  8559 consensus_queue.cc:1048] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:21.843364  8181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "663664e06ddc4e86a7055c8a134940cc"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
mode: GRACEFUL
new_leader_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:59926
I20260709 12:28:21.843933  8181 raft_consensus.cc:606] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Received request to transfer leadership to TS 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:21.846469  8181 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8da354781025485892cf19d483eb32a3"
dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
mode: GRACEFUL
new_leader_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:59926
I20260709 12:28:21.846993  8181 raft_consensus.cc:606] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 3 LEADER]: Received request to transfer leadership to TS 72fb489f0a054602bb946316bc7c187e
I20260709 12:28:21.848007 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:21.848467 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:21.848984 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:21.998716  8523 raft_consensus.cc:993] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59: : Instructing follower 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:21.999198  8559 raft_consensus.cc:1081] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Signalling peer 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:22.000531  8255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "663664e06ddc4e86a7055c8a134940cc"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:45978
I20260709 12:28:22.000928  8255 raft_consensus.cc:493] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:22.001150  8255 raft_consensus.cc:3060] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.006057  8255 raft_consensus.cc:515] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:22.007524  8255 leader_election.cc:290] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:22.008603  8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663664e06ddc4e86a7055c8a134940cc" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:22.008621  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663664e06ddc4e86a7055c8a134940cc" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:22.009243  8446 raft_consensus.cc:3060] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.009306  8181 raft_consensus.cc:3055] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:22.009722  8181 raft_consensus.cc:740] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:22.010433  8181 consensus_queue.cc:260] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:22.011416  8181 raft_consensus.cc:3060] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.014519  8446 raft_consensus.cc:2468] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 2.
I20260709 12:28:22.015533  8222 leader_election.cc:304] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:22.015792  8181 raft_consensus.cc:2468] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 2.
I20260709 12:28:22.016251  8544 raft_consensus.cc:2804] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:22.017427  8544 raft_consensus.cc:697] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:22.018343  8544 consensus_queue.cc:237] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:22.025334  8005 catalog_manager.cc:5697] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 1 to 2, leader changed from 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66) to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 2 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.148294  8523 raft_consensus.cc:993] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59: : Instructing follower 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:22.148808  8523 raft_consensus.cc:1081] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 3 LEADER]: Signalling peer 72fb489f0a054602bb946316bc7c187e to start an election
I20260709 12:28:22.150492  8255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8da354781025485892cf19d483eb32a3"
dest_uuid: "72fb489f0a054602bb946316bc7c187e"
 from {username='slave'} at 127.0.0.1:45978
I20260709 12:28:22.150892  8255 raft_consensus.cc:493] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:22.151121  8255 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:22.156652  8255 raft_consensus.cc:515] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:22.158052  8255 leader_election.cc:290] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 4 election: Requested vote from peers 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385), 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049)
I20260709 12:28:22.158782  8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67"
I20260709 12:28:22.159133  8181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8da354781025485892cf19d483eb32a3" candidate_uuid: "72fb489f0a054602bb946316bc7c187e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "712be1df6e174cd9928a200ac0a17e59"
I20260709 12:28:22.159497  8446 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:22.159793  8181 raft_consensus.cc:3055] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:22.160106  8181 raft_consensus.cc:740] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 712be1df6e174cd9928a200ac0a17e59, State: Running, Role: LEADER
I20260709 12:28:22.160665  8181 consensus_queue.cc:260] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:22.161618  8181 raft_consensus.cc:3060] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:22.164812  8446 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 4.
I20260709 12:28:22.165827  8222 leader_election.cc:304] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20dfbd3e445d44b48fec9c3dd8c42a67, 72fb489f0a054602bb946316bc7c187e; no voters: 
I20260709 12:28:22.165901  8181 raft_consensus.cc:2468] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72fb489f0a054602bb946316bc7c187e in term 4.
I20260709 12:28:22.166550  8544 raft_consensus.cc:2804] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:22.167030  8544 raft_consensus.cc:697] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 4 LEADER]: Becoming Leader. State: Replica: 72fb489f0a054602bb946316bc7c187e, State: Running, Role: LEADER
I20260709 12:28:22.167774  8544 consensus_queue.cc:237] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } }
I20260709 12:28:22.173853  8005 catalog_manager.cc:5697] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e reported cstate change: term changed from 3 to 4, leader changed from 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66) to 72fb489f0a054602bb946316bc7c187e (127.29.37.67). New cstate: current_term: 4 leader_uuid: "72fb489f0a054602bb946316bc7c187e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72fb489f0a054602bb946316bc7c187e" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 35273 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.594556  8446 raft_consensus.cc:1275] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:22.595832  8544 consensus_queue.cc:1048] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:22.608367  8181 raft_consensus.cc:1275] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:22.609874  8562 consensus_queue.cc:1048] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:22.689308  8053 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:22.718298  8447 raft_consensus.cc:1275] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 4 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:22.719645  8544 consensus_queue.cc:1048] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20dfbd3e445d44b48fec9c3dd8c42a67" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:22.727856  8181 raft_consensus.cc:1275] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 4 FOLLOWER]: Refusing update from remote peer 72fb489f0a054602bb946316bc7c187e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:22.729235  8562 consensus_queue.cc:1048] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [LEADER]: Connected to new peer: Peer: permanent_uuid: "712be1df6e174cd9928a200ac0a17e59" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:22.856563 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:22.885462 29845 tablet_server.cc:179] TabletServer@127.29.37.65:35385 shutting down...
I20260709 12:28:22.903508 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:22.904145 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.904727 29845 raft_consensus.cc:2243] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.905225 29845 raft_consensus.cc:2272] T 27a7c938363946bf9efcf432ea6d935a P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.907791 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.908504 29845 raft_consensus.cc:2243] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.908944 29845 raft_consensus.cc:2272] T 5f751c6cc07d44d28d60a92e61400400 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.910706 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.911173 29845 raft_consensus.cc:2243] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.911535 29845 raft_consensus.cc:2272] T 8da354781025485892cf19d483eb32a3 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.913600 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.914166 29845 raft_consensus.cc:2243] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.914665 29845 raft_consensus.cc:2272] T 663664e06ddc4e86a7055c8a134940cc P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.917384 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.917920 29845 raft_consensus.cc:2243] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.918375 29845 raft_consensus.cc:2272] T 86e376d584f94d5c84247a21cb2b0d76 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.920115 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.920610 29845 raft_consensus.cc:2243] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.921016 29845 raft_consensus.cc:2272] T 2face0b298ca4aa7a64cc288d6c5f6fa P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.922753 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.923237 29845 raft_consensus.cc:2243] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.923629 29845 raft_consensus.cc:2272] T e4d449e67e41460db81d4df8bff90f65 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.925326 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.925812 29845 raft_consensus.cc:2243] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.926208 29845 raft_consensus.cc:2272] T abcb28adc6ec45029601c575e7640ae2 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.927963 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.928419 29845 raft_consensus.cc:2243] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.928812 29845 raft_consensus.cc:2272] T 17bb19f5d45240ef8d5b1facd4c12e45 P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:22.930568 29845 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:22.931007 29845 raft_consensus.cc:2243] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:22.931442 29845 raft_consensus.cc:2272] T 7f90b970828c470d9efd8522dee3ddde P 20dfbd3e445d44b48fec9c3dd8c42a67 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:22.933212  8148 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 74 similar messages]
W20260709 12:28:22.935810  8148 consensus_peers.cc:597] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:22.959682 29845 tablet_server.cc:196] TabletServer@127.29.37.65:35385 shutdown complete.
I20260709 12:28:22.979597 29845 tablet_server.cc:179] TabletServer@127.29.37.66:0 shutting down...
W20260709 12:28:22.994783  8222 consensus_peers.cc:597] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.997288  8222 consensus_peers.cc:597] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:23.009224 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:23.010286 29845 tablet_replica.cc:333] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
W20260709 12:28:23.010838  8222 consensus_peers.cc:597] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:23.011094 29845 raft_consensus.cc:2243] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:23.011252  8222 consensus_peers.cc:597] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:23.011713 29845 raft_consensus.cc:2272] T 27a7c938363946bf9efcf432ea6d935a P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.013890 29845 tablet_replica.cc:333] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.014552 29845 raft_consensus.cc:2243] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.015306 29845 raft_consensus.cc:2272] T 17bb19f5d45240ef8d5b1facd4c12e45 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.017295 29845 tablet_replica.cc:333] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.017825 29845 raft_consensus.cc:2243] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.018664 29845 raft_consensus.cc:2272] T 5f751c6cc07d44d28d60a92e61400400 P 712be1df6e174cd9928a200ac0a17e59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.020653 29845 tablet_replica.cc:333] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.021263 29845 raft_consensus.cc:2243] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.021672 29845 raft_consensus.cc:2272] T 8da354781025485892cf19d483eb32a3 P 712be1df6e174cd9928a200ac0a17e59 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.024096 29845 tablet_replica.cc:333] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.024636 29845 raft_consensus.cc:2243] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.025058 29845 raft_consensus.cc:2272] T abcb28adc6ec45029601c575e7640ae2 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.027069 29845 tablet_replica.cc:333] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.027557 29845 raft_consensus.cc:2243] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.027954 29845 raft_consensus.cc:2272] T 2face0b298ca4aa7a64cc288d6c5f6fa P 712be1df6e174cd9928a200ac0a17e59 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.029805 29845 tablet_replica.cc:333] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.030431 29845 raft_consensus.cc:2243] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.030835 29845 raft_consensus.cc:2272] T 86e376d584f94d5c84247a21cb2b0d76 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.032820 29845 tablet_replica.cc:333] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.033298 29845 raft_consensus.cc:2243] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.034037 29845 raft_consensus.cc:2272] T e4d449e67e41460db81d4df8bff90f65 P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.036099 29845 tablet_replica.cc:333] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.036615 29845 raft_consensus.cc:2243] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.037034 29845 raft_consensus.cc:2272] T 663664e06ddc4e86a7055c8a134940cc P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.039464 29845 tablet_replica.cc:333] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59: stopping tablet replica
I20260709 12:28:23.039945 29845 raft_consensus.cc:2243] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.040670 29845 raft_consensus.cc:2272] T 7f90b970828c470d9efd8522dee3ddde P 712be1df6e174cd9928a200ac0a17e59 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:23.045761  8219 consensus_peers.cc:597] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e -> Peer 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049): Couldn't send request to peer 712be1df6e174cd9928a200ac0a17e59. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:23.056121  8145 connection.cc:466] server connection from 127.0.0.1:59902 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59902 (request call id 117) could send its response
W20260709 12:28:23.059182  8219 consensus_peers.cc:597] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e -> Peer 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049): Couldn't send request to peer 712be1df6e174cd9928a200ac0a17e59. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:37049: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:23.067773  8222 consensus_peers.cc:597] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e -> Peer 20dfbd3e445d44b48fec9c3dd8c42a67 (127.29.37.65:35385): Couldn't send request to peer 20dfbd3e445d44b48fec9c3dd8c42a67. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:23.073675 29845 tablet_server.cc:196] TabletServer@127.29.37.66:0 shutdown complete.
I20260709 12:28:23.099869 29845 tablet_server.cc:179] TabletServer@127.29.37.67:0 shutting down...
W20260709 12:28:23.106921  8219 consensus_peers.cc:597] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e -> Peer 712be1df6e174cd9928a200ac0a17e59 (127.29.37.66:37049): Couldn't send request to peer 712be1df6e174cd9928a200ac0a17e59. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:37049: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:23.127856 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:23.128579 29845 tablet_replica.cc:333] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.129212 29845 raft_consensus.cc:2243] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.129640 29845 raft_consensus.cc:2272] T 17bb19f5d45240ef8d5b1facd4c12e45 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.131870 29845 tablet_replica.cc:333] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.132395 29845 raft_consensus.cc:2243] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.132786 29845 raft_consensus.cc:2272] T 5f751c6cc07d44d28d60a92e61400400 P 72fb489f0a054602bb946316bc7c187e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.134833 29845 tablet_replica.cc:333] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.135306 29845 raft_consensus.cc:2243] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.135988 29845 raft_consensus.cc:2272] T 27a7c938363946bf9efcf432ea6d935a P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.137909 29845 tablet_replica.cc:333] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.138458 29845 raft_consensus.cc:2243] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.139175 29845 raft_consensus.cc:2272] T 8da354781025485892cf19d483eb32a3 P 72fb489f0a054602bb946316bc7c187e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.141538 29845 tablet_replica.cc:333] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.142019 29845 raft_consensus.cc:2243] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.142768 29845 raft_consensus.cc:2272] T abcb28adc6ec45029601c575e7640ae2 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.144649 29845 tablet_replica.cc:333] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.145129 29845 raft_consensus.cc:2243] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.145474 29845 raft_consensus.cc:2272] T 7f90b970828c470d9efd8522dee3ddde P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.147459 29845 tablet_replica.cc:333] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.147912 29845 raft_consensus.cc:2243] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.148563 29845 raft_consensus.cc:2272] T 2face0b298ca4aa7a64cc288d6c5f6fa P 72fb489f0a054602bb946316bc7c187e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.150575 29845 tablet_replica.cc:333] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.151036 29845 raft_consensus.cc:2243] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.151697 29845 raft_consensus.cc:2272] T 86e376d584f94d5c84247a21cb2b0d76 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.153846 29845 tablet_replica.cc:333] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.154361 29845 raft_consensus.cc:2243] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:23.154718 29845 raft_consensus.cc:2272] T e4d449e67e41460db81d4df8bff90f65 P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.156701 29845 tablet_replica.cc:333] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e: stopping tablet replica
I20260709 12:28:23.157107 29845 raft_consensus.cc:2243] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.157831 29845 raft_consensus.cc:2272] T 663664e06ddc4e86a7055c8a134940cc P 72fb489f0a054602bb946316bc7c187e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.191656 29845 tablet_server.cc:196] TabletServer@127.29.37.67:0 shutdown complete.
I20260709 12:28:23.216675 29845 master.cc:562] Master@127.29.37.126:37501 shutting down...
I20260709 12:28:23.241078 29845 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:23.241616 29845 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:23.241894 29845 tablet_replica.cc:333] T 00000000000000000000000000000000 P 267aa51e8905414787e6c01124ee21d4: stopping tablet replica
I20260709 12:28:23.261175 29845 master.cc:584] Master@127.29.37.126:37501 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12770 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:28:23.303622 29845 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.29.37.126:36177
I20260709 12:28:23.304796 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:23.310909  8583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:23.311107  8584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:23.311221  8586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:23.312077 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:23.312927 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:23.313091 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:23.313216 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600103313205 us; error 0 us; skew 500 ppm
I20260709 12:28:23.313665 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:23.315750 29845 webserver.cc:533] Webserver started at http://127.29.37.126:36283/ using document root <none> and password file <none>
I20260709 12:28:23.316200 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:23.316354 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:23.316566 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:23.317571 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/master-0-root/instance:
uuid: "3ba9fd274fa242a5acc209b4a5d363df"
format_stamp: "Formatted at 2026-07-09 12:28:23 on dist-test-slave-g1m6"
I20260709 12:28:23.321705 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 12:28:23.324772  8591 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.325522 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:28:23.325774 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/master-0-root
uuid: "3ba9fd274fa242a5acc209b4a5d363df"
format_stamp: "Formatted at 2026-07-09 12:28:23 on dist-test-slave-g1m6"
I20260709 12:28:23.326009 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:23.356933 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:23.357995 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:23.395049 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.126:36177
I20260709 12:28:23.395154  8642 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.126:36177 every 8 connection(s)
I20260709 12:28:23.398886  8643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.410084  8643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df: Bootstrap starting.
I20260709 12:28:23.414999  8643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.419237  8643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df: No bootstrap required, opened a new log
I20260709 12:28:23.421415  8643 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba9fd274fa242a5acc209b4a5d363df" member_type: VOTER }
I20260709 12:28:23.421793  8643 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.422061  8643 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba9fd274fa242a5acc209b4a5d363df, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.422667  8643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba9fd274fa242a5acc209b4a5d363df" member_type: VOTER }
I20260709 12:28:23.423141  8643 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:23.423364  8643 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:23.423636  8643 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:23.428334  8643 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba9fd274fa242a5acc209b4a5d363df" member_type: VOTER }
I20260709 12:28:23.428823  8643 leader_election.cc:304] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ba9fd274fa242a5acc209b4a5d363df; no voters: 
I20260709 12:28:23.429916  8643 leader_election.cc:290] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:23.430285  8646 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:23.431663  8646 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 1 LEADER]: Becoming Leader. State: Replica: 3ba9fd274fa242a5acc209b4a5d363df, State: Running, Role: LEADER
I20260709 12:28:23.432307  8646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba9fd274fa242a5acc209b4a5d363df" member_type: VOTER }
I20260709 12:28:23.432723  8643 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:23.436141  8647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ba9fd274fa242a5acc209b4a5d363df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba9fd274fa242a5acc209b4a5d363df" member_type: VOTER } }
I20260709 12:28:23.436815  8647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:23.441114  8648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ba9fd274fa242a5acc209b4a5d363df. Latest consensus state: current_term: 1 leader_uuid: "3ba9fd274fa242a5acc209b4a5d363df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba9fd274fa242a5acc209b4a5d363df" member_type: VOTER } }
I20260709 12:28:23.441851  8648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:23.447782  8653 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:23.448002 29845 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:23.453125  8653 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:23.461484  8653 catalog_manager.cc:1383] Generated new cluster ID: 7efaafbc230b41ecaff39cba7de90609
I20260709 12:28:23.461720  8653 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:23.477983  8653 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:23.479300  8653 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:23.494463  8653 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df: Generated new TSK 0
I20260709 12:28:23.495010  8653 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:23.514428 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:23.520192  8664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:23.521554  8665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:23.522756  8667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:23.523648 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:23.524576 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:23.524782 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:23.524953 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600103524935 us; error 0 us; skew 500 ppm
I20260709 12:28:23.525496 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:23.528038 29845 webserver.cc:533] Webserver started at http://127.29.37.65:42327/ using document root <none> and password file <none>
I20260709 12:28:23.528512 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:23.528713 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:23.528975 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:23.530055 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-0-root/instance:
uuid: "69cd28922ac1481894b3a4e82609f71a"
format_stamp: "Formatted at 2026-07-09 12:28:23 on dist-test-slave-g1m6"
I20260709 12:28:23.534526 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:23.537664  8672 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.538482 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:23.538780 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "69cd28922ac1481894b3a4e82609f71a"
format_stamp: "Formatted at 2026-07-09 12:28:23 on dist-test-slave-g1m6"
I20260709 12:28:23.539065 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:23.564189 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:23.565508 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:23.567081 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:23.569289 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:23.569517 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.569774 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:23.569954 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.610342 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:35985
I20260709 12:28:23.610468  8734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:35985 every 8 connection(s)
I20260709 12:28:23.615767 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:23.622643  8739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:23.623035  8740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:23.627377  8742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:23.628259 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:23.628562  8735 heartbeater.cc:344] Connected to a master server at 127.29.37.126:36177
I20260709 12:28:23.628988  8735 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:23.629460 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:23.629730 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:23.629752  8735 heartbeater.cc:507] Master 127.29.37.126:36177 requested a full tablet report, sending...
I20260709 12:28:23.629933 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600103629915 us; error 0 us; skew 500 ppm
I20260709 12:28:23.630673 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:23.632108  8608 ts_manager.cc:194] Registered new tserver with Master: 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985)
I20260709 12:28:23.633116 29845 webserver.cc:533] Webserver started at http://127.29.37.66:37561/ using document root <none> and password file <none>
I20260709 12:28:23.633646 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:23.633862 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:23.634132 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:23.634140  8608 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54736
I20260709 12:28:23.635547 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-1-root/instance:
uuid: "5aa9cc533d124a11b464147f0dc699ed"
format_stamp: "Formatted at 2026-07-09 12:28:23 on dist-test-slave-g1m6"
I20260709 12:28:23.639947 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:28:23.643378  8747 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.644196 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:23.644500 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-1-root
uuid: "5aa9cc533d124a11b464147f0dc699ed"
format_stamp: "Formatted at 2026-07-09 12:28:23 on dist-test-slave-g1m6"
I20260709 12:28:23.644802 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:23.664570 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:23.665807 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:23.667347 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:23.669572 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:23.669806 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.670047 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:23.670222 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.710469 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.66:45303
I20260709 12:28:23.710572  8809 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.66:45303 every 8 connection(s)
I20260709 12:28:23.715312 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:23.722494  8813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:23.723261  8814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:23.726764  8816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:23.727582 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:23.727749  8810 heartbeater.cc:344] Connected to a master server at 127.29.37.126:36177
I20260709 12:28:23.728293  8810 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:23.728658 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:23.728937 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:23.729059  8810 heartbeater.cc:507] Master 127.29.37.126:36177 requested a full tablet report, sending...
I20260709 12:28:23.729128 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600103729112 us; error 0 us; skew 500 ppm
I20260709 12:28:23.730075 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:23.731601  8608 ts_manager.cc:194] Registered new tserver with Master: 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303)
I20260709 12:28:23.732863 29845 webserver.cc:533] Webserver started at http://127.29.37.67:35745/ using document root <none> and password file <none>
I20260709 12:28:23.733512 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:23.733593  8608 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54748
I20260709 12:28:23.733777 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:23.734237 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:23.735781 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-2-root/instance:
uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
format_stamp: "Formatted at 2026-07-09 12:28:23 on dist-test-slave-g1m6"
I20260709 12:28:23.740269 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:28:23.743582  8821 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.744305 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:23.744606 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-2-root
uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
format_stamp: "Formatted at 2026-07-09 12:28:23 on dist-test-slave-g1m6"
I20260709 12:28:23.744875 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599734091681-29845-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:23.765669 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:23.766875 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:23.768314 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:23.770609 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:23.770839 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.771075 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:23.771246 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:23.810067 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.67:45271
I20260709 12:28:23.810182  8883 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.67:45271 every 8 connection(s)
I20260709 12:28:23.823293  8884 heartbeater.cc:344] Connected to a master server at 127.29.37.126:36177
I20260709 12:28:23.823689  8884 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:23.824453  8884 heartbeater.cc:507] Master 127.29.37.126:36177 requested a full tablet report, sending...
I20260709 12:28:23.826296  8608 ts_manager.cc:194] Registered new tserver with Master: d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67:45271)
I20260709 12:28:23.826644 29845 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013350024s
I20260709 12:28:23.828051  8608 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54758
I20260709 12:28:23.851946  8608 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54774:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:23.853996  8608 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:23.903400  8849 tablet_service.cc:1511] Processing CreateTablet for tablet a469562fdc7e44278ce957aafc5e7d7c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4ff48d4dafd841c1b2010d05fc7cac6d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:23.903795  8848 tablet_service.cc:1511] Processing CreateTablet for tablet 294b9a834d2547348bc161068bbc3891 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4ff48d4dafd841c1b2010d05fc7cac6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:23.904992  8849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a469562fdc7e44278ce957aafc5e7d7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.905202  8774 tablet_service.cc:1511] Processing CreateTablet for tablet 294b9a834d2547348bc161068bbc3891 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4ff48d4dafd841c1b2010d05fc7cac6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:23.906561  8774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 294b9a834d2547348bc161068bbc3891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.906373  8775 tablet_service.cc:1511] Processing CreateTablet for tablet a469562fdc7e44278ce957aafc5e7d7c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4ff48d4dafd841c1b2010d05fc7cac6d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:23.908004  8775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a469562fdc7e44278ce957aafc5e7d7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.908876  8699 tablet_service.cc:1511] Processing CreateTablet for tablet 294b9a834d2547348bc161068bbc3891 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4ff48d4dafd841c1b2010d05fc7cac6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:23.910179  8699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 294b9a834d2547348bc161068bbc3891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.910594  8848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 294b9a834d2547348bc161068bbc3891. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.907275  8700 tablet_service.cc:1511] Processing CreateTablet for tablet a469562fdc7e44278ce957aafc5e7d7c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4ff48d4dafd841c1b2010d05fc7cac6d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:23.911706  8700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a469562fdc7e44278ce957aafc5e7d7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:23.936950  8900 tablet_bootstrap.cc:492] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb: Bootstrap starting.
I20260709 12:28:23.944818  8900 tablet_bootstrap.cc:654] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.951119  8901 tablet_bootstrap.cc:492] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a: Bootstrap starting.
I20260709 12:28:23.954476  8900 tablet_bootstrap.cc:492] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb: No bootstrap required, opened a new log
I20260709 12:28:23.955113  8900 ts_tablet_manager.cc:1403] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:28:23.962248  8899 tablet_bootstrap.cc:492] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed: Bootstrap starting.
I20260709 12:28:23.962548  8900 raft_consensus.cc:359] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:23.963367  8900 raft_consensus.cc:385] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.963670  8900 raft_consensus.cc:740] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.964617  8901 tablet_bootstrap.cc:654] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.964437  8900 consensus_queue.cc:260] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:23.967422  8884 heartbeater.cc:499] Master 127.29.37.126:36177 was elected leader, sending a full tablet report...
I20260709 12:28:23.967818  8900 ts_tablet_manager.cc:1434] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:28:23.968709  8900 tablet_bootstrap.cc:492] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Bootstrap starting.
I20260709 12:28:23.970193  8899 tablet_bootstrap.cc:654] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.975296  8900 tablet_bootstrap.cc:654] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:23.982113  8901 tablet_bootstrap.cc:492] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a: No bootstrap required, opened a new log
I20260709 12:28:23.982825  8901 ts_tablet_manager.cc:1403] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a: Time spent bootstrapping tablet: real 0.032s	user 0.006s	sys 0.013s
I20260709 12:28:23.986917  8901 raft_consensus.cc:359] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:23.987954  8901 raft_consensus.cc:385] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.988333  8901 raft_consensus.cc:740] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69cd28922ac1481894b3a4e82609f71a, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.990125  8901 consensus_queue.cc:260] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:23.992165  8899 tablet_bootstrap.cc:492] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed: No bootstrap required, opened a new log
I20260709 12:28:23.992676  8735 heartbeater.cc:499] Master 127.29.37.126:36177 was elected leader, sending a full tablet report...
I20260709 12:28:23.992791  8899 ts_tablet_manager.cc:1403] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed: Time spent bootstrapping tablet: real 0.031s	user 0.021s	sys 0.009s
I20260709 12:28:23.993160  8900 tablet_bootstrap.cc:492] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb: No bootstrap required, opened a new log
I20260709 12:28:23.993266  8901 ts_tablet_manager.cc:1434] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:28:23.993736  8900 ts_tablet_manager.cc:1403] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Time spent bootstrapping tablet: real 0.025s	user 0.001s	sys 0.021s
I20260709 12:28:23.996423  8899 raft_consensus.cc:359] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:23.996773  8900 raft_consensus.cc:359] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:23.997277  8899 raft_consensus.cc:385] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.997520  8900 raft_consensus.cc:385] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:23.997874  8900 raft_consensus.cc:740] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.997733  8899 raft_consensus.cc:740] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa9cc533d124a11b464147f0dc699ed, State: Initialized, Role: FOLLOWER
I20260709 12:28:23.998741  8900 consensus_queue.cc:260] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.000969  8900 ts_tablet_manager.cc:1434] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:23.999136  8899 consensus_queue.cc:260] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.004314  8901 tablet_bootstrap.cc:492] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a: Bootstrap starting.
I20260709 12:28:24.004917  8810 heartbeater.cc:499] Master 127.29.37.126:36177 was elected leader, sending a full tablet report...
I20260709 12:28:24.005594  8899 ts_tablet_manager.cc:1434] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:28:24.006678  8899 tablet_bootstrap.cc:492] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed: Bootstrap starting.
I20260709 12:28:24.013125  8899 tablet_bootstrap.cc:654] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.013176  8901 tablet_bootstrap.cc:654] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.018906  8903 raft_consensus.cc:493] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.019609  8903 raft_consensus.cc:515] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.024506  8899 tablet_bootstrap.cc:492] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed: No bootstrap required, opened a new log
I20260709 12:28:24.025024  8899 ts_tablet_manager.cc:1403] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:28:24.025835  8901 tablet_bootstrap.cc:492] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a: No bootstrap required, opened a new log
I20260709 12:28:24.026384  8901 ts_tablet_manager.cc:1403] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 12:28:24.028067  8899 raft_consensus.cc:359] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.028822  8899 raft_consensus.cc:385] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.029115  8899 raft_consensus.cc:740] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa9cc533d124a11b464147f0dc699ed, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.029299  8901 raft_consensus.cc:359] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.030062  8901 raft_consensus.cc:385] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.029891  8899 consensus_queue.cc:260] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.032341  8899 ts_tablet_manager.cc:1434] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:28:24.030853  8903 leader_election.cc:290] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985), 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303)
I20260709 12:28:24.030386  8901 raft_consensus.cc:740] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69cd28922ac1481894b3a4e82609f71a, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.034003  8901 consensus_queue.cc:260] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.036340  8710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a469562fdc7e44278ce957aafc5e7d7c" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cd28922ac1481894b3a4e82609f71a" is_pre_election: true
I20260709 12:28:24.037025  8710 raft_consensus.cc:2468] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 0.
I20260709 12:28:24.038442  8825 leader_election.cc:304] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cd28922ac1481894b3a4e82609f71a, d1ff59f123834cf9bf45ab6dcd5ff3eb; no voters: 
I20260709 12:28:24.039347  8903 raft_consensus.cc:2804] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.039523  8901 ts_tablet_manager.cc:1434] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:28:24.039731  8903 raft_consensus.cc:493] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.040251  8903 raft_consensus.cc:3060] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.045310  8785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a469562fdc7e44278ce957aafc5e7d7c" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa9cc533d124a11b464147f0dc699ed" is_pre_election: true
I20260709 12:28:24.046057  8785 raft_consensus.cc:2468] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 0.
I20260709 12:28:24.048611  8903 raft_consensus.cc:515] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.050513  8903 leader_election.cc:290] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 1 election: Requested vote from peers 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985), 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303)
I20260709 12:28:24.051360  8710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a469562fdc7e44278ce957aafc5e7d7c" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cd28922ac1481894b3a4e82609f71a"
I20260709 12:28:24.051410  8785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a469562fdc7e44278ce957aafc5e7d7c" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa9cc533d124a11b464147f0dc699ed"
I20260709 12:28:24.052183  8710 raft_consensus.cc:3060] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.052314  8785 raft_consensus.cc:3060] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.057456  8710 raft_consensus.cc:2468] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 1.
I20260709 12:28:24.057565  8785 raft_consensus.cc:2468] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 1.
I20260709 12:28:24.058511  8825 leader_election.cc:304] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cd28922ac1481894b3a4e82609f71a, d1ff59f123834cf9bf45ab6dcd5ff3eb; no voters: 
I20260709 12:28:24.059201  8903 raft_consensus.cc:2804] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.060199  8903 raft_consensus.cc:697] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Becoming Leader. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Running, Role: LEADER
I20260709 12:28:24.060998  8903 consensus_queue.cc:237] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.069555  8606 catalog_manager.cc:5697] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb reported cstate change: term changed from 0 to 1, leader changed from <none> to d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67). New cstate: current_term: 1 leader_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.104856  8903 raft_consensus.cc:493] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.105423  8903 raft_consensus.cc:515] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.107210  8903 leader_election.cc:290] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985), 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303)
I20260709 12:28:24.107940  8710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294b9a834d2547348bc161068bbc3891" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cd28922ac1481894b3a4e82609f71a" is_pre_election: true
I20260709 12:28:24.108230  8785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294b9a834d2547348bc161068bbc3891" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa9cc533d124a11b464147f0dc699ed" is_pre_election: true
I20260709 12:28:24.108680  8710 raft_consensus.cc:2468] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 0.
I20260709 12:28:24.108866  8785 raft_consensus.cc:2468] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 0.
I20260709 12:28:24.109748  8825 leader_election.cc:304] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cd28922ac1481894b3a4e82609f71a, d1ff59f123834cf9bf45ab6dcd5ff3eb; no voters: 
I20260709 12:28:24.110471  8903 raft_consensus.cc:2804] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.110738  8903 raft_consensus.cc:493] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.110949  8903 raft_consensus.cc:3060] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.116752  8903 raft_consensus.cc:515] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.118359  8903 leader_election.cc:290] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 1 election: Requested vote from peers 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985), 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303)
I20260709 12:28:24.119050  8710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294b9a834d2547348bc161068bbc3891" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cd28922ac1481894b3a4e82609f71a"
I20260709 12:28:24.119297  8785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294b9a834d2547348bc161068bbc3891" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa9cc533d124a11b464147f0dc699ed"
I20260709 12:28:24.119614  8710 raft_consensus.cc:3060] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.119874  8785 raft_consensus.cc:3060] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.124440  8710 raft_consensus.cc:2468] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 1.
I20260709 12:28:24.124876  8785 raft_consensus.cc:2468] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 1.
I20260709 12:28:24.125471  8825 leader_election.cc:304] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cd28922ac1481894b3a4e82609f71a, d1ff59f123834cf9bf45ab6dcd5ff3eb; no voters: 
I20260709 12:28:24.126338  8903 raft_consensus.cc:2804] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.126683  8903 raft_consensus.cc:697] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Becoming Leader. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Running, Role: LEADER
I20260709 12:28:24.127585  8903 consensus_queue.cc:237] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.134488  8606 catalog_manager.cc:5697] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb reported cstate change: term changed from 0 to 1, leader changed from <none> to d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67). New cstate: current_term: 1 leader_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:24.168563  8606 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54776:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:24.170573  8606 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:24.192999  8700 tablet_service.cc:1511] Processing CreateTablet for tablet c7ff6e55aef24ff5a5060ae975b236e1 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e9df4432c329488ba0f915dcf3bb424f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:24.194784  8700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7ff6e55aef24ff5a5060ae975b236e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:24.194492  8775 tablet_service.cc:1511] Processing CreateTablet for tablet c7ff6e55aef24ff5a5060ae975b236e1 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e9df4432c329488ba0f915dcf3bb424f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:24.195817  8775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7ff6e55aef24ff5a5060ae975b236e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:24.196731  8849 tablet_service.cc:1511] Processing CreateTablet for tablet c7ff6e55aef24ff5a5060ae975b236e1 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e9df4432c329488ba0f915dcf3bb424f]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:24.197979  8849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7ff6e55aef24ff5a5060ae975b236e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:24.198858  8699 tablet_service.cc:1511] Processing CreateTablet for tablet b4957601cc8e4de4a0c8c77dafb02122 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e9df4432c329488ba0f915dcf3bb424f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:24.200126  8699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4957601cc8e4de4a0c8c77dafb02122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:24.201731  8848 tablet_service.cc:1511] Processing CreateTablet for tablet b4957601cc8e4de4a0c8c77dafb02122 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e9df4432c329488ba0f915dcf3bb424f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:24.201911  8774 tablet_service.cc:1511] Processing CreateTablet for tablet b4957601cc8e4de4a0c8c77dafb02122 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e9df4432c329488ba0f915dcf3bb424f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:24.203006  8848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4957601cc8e4de4a0c8c77dafb02122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:24.203023  8774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4957601cc8e4de4a0c8c77dafb02122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:24.211584  8901 tablet_bootstrap.cc:492] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a: Bootstrap starting.
I20260709 12:28:24.216660  8900 tablet_bootstrap.cc:492] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Bootstrap starting.
I20260709 12:28:24.217993  8901 tablet_bootstrap.cc:654] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.223735  8900 tablet_bootstrap.cc:654] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.235610  8899 tablet_bootstrap.cc:492] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed: Bootstrap starting.
I20260709 12:28:24.235704  8901 tablet_bootstrap.cc:492] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a: No bootstrap required, opened a new log
I20260709 12:28:24.236438  8901 ts_tablet_manager.cc:1403] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.012s
I20260709 12:28:24.239473  8901 raft_consensus.cc:359] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.240223  8901 raft_consensus.cc:385] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.240528  8901 raft_consensus.cc:740] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69cd28922ac1481894b3a4e82609f71a, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.241495  8901 consensus_queue.cc:260] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.244259  8900 tablet_bootstrap.cc:492] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb: No bootstrap required, opened a new log
I20260709 12:28:24.244818  8900 ts_tablet_manager.cc:1403] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.004s
I20260709 12:28:24.245378  8899 tablet_bootstrap.cc:654] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.246402  8901 ts_tablet_manager.cc:1434] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:28:24.247444  8901 tablet_bootstrap.cc:492] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a: Bootstrap starting.
I20260709 12:28:24.247998  8900 raft_consensus.cc:359] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.248739  8900 raft_consensus.cc:385] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.249033  8900 raft_consensus.cc:740] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.249954  8900 consensus_queue.cc:260] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.255920  8901 tablet_bootstrap.cc:654] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.255970  8900 ts_tablet_manager.cc:1434] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:28:24.257390  8900 tablet_bootstrap.cc:492] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Bootstrap starting.
I20260709 12:28:24.262373  8899 tablet_bootstrap.cc:492] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed: No bootstrap required, opened a new log
I20260709 12:28:24.262975  8899 ts_tablet_manager.cc:1403] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed: Time spent bootstrapping tablet: real 0.028s	user 0.004s	sys 0.015s
I20260709 12:28:24.266098  8899 raft_consensus.cc:359] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.266650  8900 tablet_bootstrap.cc:654] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.266901  8899 raft_consensus.cc:385] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.267246  8899 raft_consensus.cc:740] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa9cc533d124a11b464147f0dc699ed, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.268096  8899 consensus_queue.cc:260] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.268695  8901 tablet_bootstrap.cc:492] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a: No bootstrap required, opened a new log
I20260709 12:28:24.269282  8901 ts_tablet_manager.cc:1403] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.005s
I20260709 12:28:24.272336  8901 raft_consensus.cc:359] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.273029  8901 raft_consensus.cc:385] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.273110  8899 ts_tablet_manager.cc:1434] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:28:24.273391  8901 raft_consensus.cc:740] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69cd28922ac1481894b3a4e82609f71a, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.274369  8899 tablet_bootstrap.cc:492] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed: Bootstrap starting.
I20260709 12:28:24.274336  8901 consensus_queue.cc:260] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.276719  8901 ts_tablet_manager.cc:1434] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:24.278409  8900 tablet_bootstrap.cc:492] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb: No bootstrap required, opened a new log
I20260709 12:28:24.278913  8900 ts_tablet_manager.cc:1403] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:28:24.281929  8899 tablet_bootstrap.cc:654] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:24.281818  8900 raft_consensus.cc:359] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.282594  8900 raft_consensus.cc:385] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.282912  8900 raft_consensus.cc:740] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.284349  8900 consensus_queue.cc:260] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.286666  8900 ts_tablet_manager.cc:1434] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:24.291172  8899 tablet_bootstrap.cc:492] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed: No bootstrap required, opened a new log
I20260709 12:28:24.291715  8899 ts_tablet_manager.cc:1403] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 12:28:24.293919  8899 raft_consensus.cc:359] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.294534  8899 raft_consensus.cc:385] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:24.294833  8899 raft_consensus.cc:740] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa9cc533d124a11b464147f0dc699ed, State: Initialized, Role: FOLLOWER
I20260709 12:28:24.295558  8899 consensus_queue.cc:260] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.297860  8899 ts_tablet_manager.cc:1434] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:24.421556  8908 raft_consensus.cc:493] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.422101  8908 raft_consensus.cc:515] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.424256  8908 leader_election.cc:290] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985), d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67:45271)
I20260709 12:28:24.435497  8710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122" candidate_uuid: "5aa9cc533d124a11b464147f0dc699ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cd28922ac1481894b3a4e82609f71a" is_pre_election: true
I20260709 12:28:24.436216  8710 raft_consensus.cc:2468] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa9cc533d124a11b464147f0dc699ed in term 0.
I20260709 12:28:24.436300  8859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122" candidate_uuid: "5aa9cc533d124a11b464147f0dc699ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" is_pre_election: true
I20260709 12:28:24.436986  8859 raft_consensus.cc:2468] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa9cc533d124a11b464147f0dc699ed in term 0.
I20260709 12:28:24.437358  8751 leader_election.cc:304] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa9cc533d124a11b464147f0dc699ed, 69cd28922ac1481894b3a4e82609f71a; no voters: 
I20260709 12:28:24.438199  8908 raft_consensus.cc:2804] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.438640  8908 raft_consensus.cc:493] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.438920  8908 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:24.442028  8656 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:24.444837  8908 raft_consensus.cc:515] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.446501  8908 leader_election.cc:290] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [CANDIDATE]: Term 1 election: Requested vote from peers 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985), d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67:45271)
I20260709 12:28:24.447257  8710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122" candidate_uuid: "5aa9cc533d124a11b464147f0dc699ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69cd28922ac1481894b3a4e82609f71a"
I20260709 12:28:24.447742  8859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122" candidate_uuid: "5aa9cc533d124a11b464147f0dc699ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
I20260709 12:28:24.447881  8710 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.448304  8859 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.453003  8710 raft_consensus.cc:2468] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa9cc533d124a11b464147f0dc699ed in term 1.
I20260709 12:28:24.453473  8859 raft_consensus.cc:2468] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa9cc533d124a11b464147f0dc699ed in term 1.
I20260709 12:28:24.454084  8751 leader_election.cc:304] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa9cc533d124a11b464147f0dc699ed, 69cd28922ac1481894b3a4e82609f71a; no voters: 
I20260709 12:28:24.454900  8908 raft_consensus.cc:2804] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.455688  8908 raft_consensus.cc:697] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 1 LEADER]: Becoming Leader. State: Replica: 5aa9cc533d124a11b464147f0dc699ed, State: Running, Role: LEADER
I20260709 12:28:24.456475  8908 consensus_queue.cc:237] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.462826  8606 catalog_manager.cc:5697] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66). New cstate: current_term: 1 leader_uuid: "5aa9cc533d124a11b464147f0dc699ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.502043  8907 raft_consensus.cc:493] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.502691  8907 raft_consensus.cc:515] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.504835  8907 leader_election.cc:290] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303), d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67:45271)
I20260709 12:28:24.514532  8785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ff6e55aef24ff5a5060ae975b236e1" candidate_uuid: "69cd28922ac1481894b3a4e82609f71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa9cc533d124a11b464147f0dc699ed" is_pre_election: true
I20260709 12:28:24.515118  8859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ff6e55aef24ff5a5060ae975b236e1" candidate_uuid: "69cd28922ac1481894b3a4e82609f71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" is_pre_election: true
I20260709 12:28:24.515256  8785 raft_consensus.cc:2468] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69cd28922ac1481894b3a4e82609f71a in term 0.
I20260709 12:28:24.515938  8859 raft_consensus.cc:2468] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69cd28922ac1481894b3a4e82609f71a in term 0.
I20260709 12:28:24.516574  8674 leader_election.cc:304] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa9cc533d124a11b464147f0dc699ed, 69cd28922ac1481894b3a4e82609f71a; no voters: 
I20260709 12:28:24.517453  8907 raft_consensus.cc:2804] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:24.517772  8907 raft_consensus.cc:493] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.518038  8907 raft_consensus.cc:3060] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.523156  8907 raft_consensus.cc:515] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.524967  8907 leader_election.cc:290] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [CANDIDATE]: Term 1 election: Requested vote from peers 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303), d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67:45271)
I20260709 12:28:24.525498  8785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ff6e55aef24ff5a5060ae975b236e1" candidate_uuid: "69cd28922ac1481894b3a4e82609f71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa9cc533d124a11b464147f0dc699ed"
I20260709 12:28:24.526073  8859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7ff6e55aef24ff5a5060ae975b236e1" candidate_uuid: "69cd28922ac1481894b3a4e82609f71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
I20260709 12:28:24.526098  8785 raft_consensus.cc:3060] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.526750  8859 raft_consensus.cc:3060] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:24.531350  8785 raft_consensus.cc:2468] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69cd28922ac1481894b3a4e82609f71a in term 1.
I20260709 12:28:24.531587  8859 raft_consensus.cc:2468] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69cd28922ac1481894b3a4e82609f71a in term 1.
I20260709 12:28:24.532405  8674 leader_election.cc:304] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa9cc533d124a11b464147f0dc699ed, 69cd28922ac1481894b3a4e82609f71a; no voters: 
I20260709 12:28:24.533241  8907 raft_consensus.cc:2804] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:24.534403  8907 raft_consensus.cc:697] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 1 LEADER]: Becoming Leader. State: Replica: 69cd28922ac1481894b3a4e82609f71a, State: Running, Role: LEADER
I20260709 12:28:24.535578  8907 consensus_queue.cc:237] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.542094  8606 catalog_manager.cc:5697] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a reported cstate change: term changed from 0 to 1, leader changed from <none> to 69cd28922ac1481894b3a4e82609f71a (127.29.37.65). New cstate: current_term: 1 leader_uuid: "69cd28922ac1481894b3a4e82609f71a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.551766  8903 consensus_queue.cc:1048] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.563112  8903 consensus_queue.cc:1048] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.621068  8859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a469562fdc7e44278ce957aafc5e7d7c"
dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
mode: GRACEFUL
new_leader_uuid: "69cd28922ac1481894b3a4e82609f71a"
 from {username='slave'} at 127.0.0.1:49610
I20260709 12:28:24.621539  8859 raft_consensus.cc:606] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Received request to transfer leadership to TS 69cd28922ac1481894b3a4e82609f71a
I20260709 12:28:24.626277  8859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "294b9a834d2547348bc161068bbc3891"
dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
mode: GRACEFUL
new_leader_uuid: "5aa9cc533d124a11b464147f0dc699ed"
 from {username='slave'} at 127.0.0.1:49610
I20260709 12:28:24.626690  8859 raft_consensus.cc:606] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Received request to transfer leadership to TS 5aa9cc533d124a11b464147f0dc699ed
I20260709 12:28:24.641911  8785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122"
dest_uuid: "5aa9cc533d124a11b464147f0dc699ed"
mode: GRACEFUL
new_leader_uuid: "69cd28922ac1481894b3a4e82609f71a"
 from {username='slave'} at 127.0.0.1:45024
I20260709 12:28:24.642541  8785 raft_consensus.cc:606] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 1 LEADER]: Received request to transfer leadership to TS 69cd28922ac1481894b3a4e82609f71a
I20260709 12:28:24.655660  8903 consensus_queue.cc:1048] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.663671  8912 raft_consensus.cc:993] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb: : Instructing follower 69cd28922ac1481894b3a4e82609f71a to start an election
I20260709 12:28:24.664151  8912 raft_consensus.cc:1081] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Signalling peer 69cd28922ac1481894b3a4e82609f71a to start an election
I20260709 12:28:24.666054  8709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a469562fdc7e44278ce957aafc5e7d7c"
dest_uuid: "69cd28922ac1481894b3a4e82609f71a"
 from {username='slave'} at 127.0.0.1:38776
I20260709 12:28:24.666687  8709 raft_consensus.cc:493] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:24.666931  8709 raft_consensus.cc:3060] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:24.669750  8912 consensus_queue.cc:1048] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.674801  8709 raft_consensus.cc:515] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.676491  8710 raft_consensus.cc:1240] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Rejecting Update request from peer d1ff59f123834cf9bf45ab6dcd5ff3eb for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:24.678205  8859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a469562fdc7e44278ce957aafc5e7d7c" candidate_uuid: "69cd28922ac1481894b3a4e82609f71a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
I20260709 12:28:24.678843  8859 raft_consensus.cc:3055] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:24.678996  8784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a469562fdc7e44278ce957aafc5e7d7c" candidate_uuid: "69cd28922ac1481894b3a4e82609f71a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5aa9cc533d124a11b464147f0dc699ed"
I20260709 12:28:24.679215  8859 raft_consensus.cc:740] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Running, Role: LEADER
I20260709 12:28:24.679700  8784 raft_consensus.cc:3060] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:24.680239  8859 consensus_queue.cc:260] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.681495  8859 raft_consensus.cc:3060] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:24.687625  8784 raft_consensus.cc:2468] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69cd28922ac1481894b3a4e82609f71a in term 2.
I20260709 12:28:24.688977  8674 leader_election.cc:304] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa9cc533d124a11b464147f0dc699ed, 69cd28922ac1481894b3a4e82609f71a; no voters: 
I20260709 12:28:24.689263  8859 raft_consensus.cc:2468] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69cd28922ac1481894b3a4e82609f71a in term 2.
I20260709 12:28:24.689926  8907 raft_consensus.cc:2804] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:24.690459  8907 raft_consensus.cc:697] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 2 LEADER]: Becoming Leader. State: Replica: 69cd28922ac1481894b3a4e82609f71a, State: Running, Role: LEADER
I20260709 12:28:24.691289  8907 consensus_queue.cc:237] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } }
I20260709 12:28:24.694844  8709 leader_election.cc:290] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [CANDIDATE]: Term 2 election: Requested vote from peers d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67:45271), 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303)
I20260709 12:28:24.698614  8606 catalog_manager.cc:5697] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a reported cstate change: term changed from 1 to 2, leader changed from d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67) to 69cd28922ac1481894b3a4e82609f71a (127.29.37.65). New cstate: current_term: 2 leader_uuid: "69cd28922ac1481894b3a4e82609f71a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.946540  8903 raft_consensus.cc:993] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb: : Instructing follower 5aa9cc533d124a11b464147f0dc699ed to start an election
I20260709 12:28:24.947055  8912 raft_consensus.cc:1081] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Signalling peer 5aa9cc533d124a11b464147f0dc699ed to start an election
I20260709 12:28:24.947453  8908 consensus_queue.cc:1048] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:24.948657  8784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "294b9a834d2547348bc161068bbc3891"
dest_uuid: "5aa9cc533d124a11b464147f0dc699ed"
 from {username='slave'} at 127.0.0.1:45006
I20260709 12:28:24.949227  8784 raft_consensus.cc:493] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:24.949616  8784 raft_consensus.cc:3060] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:24.956550  8784 raft_consensus.cc:515] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.958705  8784 leader_election.cc:290] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [CANDIDATE]: Term 2 election: Requested vote from peers 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985), d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67:45271)
I20260709 12:28:24.960860  8858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294b9a834d2547348bc161068bbc3891" candidate_uuid: "5aa9cc533d124a11b464147f0dc699ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
I20260709 12:28:24.961516  8858 raft_consensus.cc:3055] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:24.961834  8858 raft_consensus.cc:740] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Running, Role: LEADER
I20260709 12:28:24.962677  8858 consensus_queue.cc:260] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.963248  8709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "294b9a834d2547348bc161068bbc3891" candidate_uuid: "5aa9cc533d124a11b464147f0dc699ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "69cd28922ac1481894b3a4e82609f71a"
I20260709 12:28:24.963927  8709 raft_consensus.cc:3060] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:24.964083  8858 raft_consensus.cc:3060] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:24.963171  8929 consensus_queue.cc:1048] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:28:24.970252  8709 raft_consensus.cc:2468] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa9cc533d124a11b464147f0dc699ed in term 2.
I20260709 12:28:24.971489  8751 leader_election.cc:304] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa9cc533d124a11b464147f0dc699ed, 69cd28922ac1481894b3a4e82609f71a; no voters: 
I20260709 12:28:24.972311  8908 raft_consensus.cc:2804] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:24.972815  8908 raft_consensus.cc:697] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 2 LEADER]: Becoming Leader. State: Replica: 5aa9cc533d124a11b464147f0dc699ed, State: Running, Role: LEADER
I20260709 12:28:24.973955  8858 raft_consensus.cc:2468] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa9cc533d124a11b464147f0dc699ed in term 2.
I20260709 12:28:24.973860  8908 consensus_queue.cc:237] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.980186  8908 raft_consensus.cc:993] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed: : Instructing follower 69cd28922ac1481894b3a4e82609f71a to start an election
I20260709 12:28:24.980655  8929 raft_consensus.cc:1081] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 1 LEADER]: Signalling peer 69cd28922ac1481894b3a4e82609f71a to start an election
I20260709 12:28:24.982364  8710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122"
dest_uuid: "69cd28922ac1481894b3a4e82609f71a"
 from {username='slave'} at 127.0.0.1:38780
I20260709 12:28:24.982467  8607 catalog_manager.cc:5697] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed reported cstate change: term changed from 1 to 2, leader changed from d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67) to 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66). New cstate: current_term: 2 leader_uuid: "5aa9cc533d124a11b464147f0dc699ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.984894  8710 raft_consensus.cc:493] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:24.985299  8710 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:24.990960  8710 raft_consensus.cc:515] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.993413  8710 leader_election.cc:290] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [CANDIDATE]: Term 2 election: Requested vote from peers 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303), d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67:45271)
I20260709 12:28:24.993921  8784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122" candidate_uuid: "69cd28922ac1481894b3a4e82609f71a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5aa9cc533d124a11b464147f0dc699ed"
I20260709 12:28:24.994014  8858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122" candidate_uuid: "69cd28922ac1481894b3a4e82609f71a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
I20260709 12:28:24.994704  8784 raft_consensus.cc:3055] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:24.994794  8858 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:24.995028  8784 raft_consensus.cc:740] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5aa9cc533d124a11b464147f0dc699ed, State: Running, Role: LEADER
I20260709 12:28:24.995781  8784 consensus_queue.cc:260] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:24.996718  8784 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:25.000105  8858 raft_consensus.cc:2468] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69cd28922ac1481894b3a4e82609f71a in term 2.
I20260709 12:28:25.001057  8673 leader_election.cc:304] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69cd28922ac1481894b3a4e82609f71a, d1ff59f123834cf9bf45ab6dcd5ff3eb; no voters: 
I20260709 12:28:25.001770  8907 raft_consensus.cc:2804] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:25.002023  8784 raft_consensus.cc:2468] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69cd28922ac1481894b3a4e82609f71a in term 2.
I20260709 12:28:25.002323  8907 raft_consensus.cc:697] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 2 LEADER]: Becoming Leader. State: Replica: 69cd28922ac1481894b3a4e82609f71a, State: Running, Role: LEADER
I20260709 12:28:25.003264  8907 consensus_queue.cc:237] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:25.009083  8607 catalog_manager.cc:5697] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a reported cstate change: term changed from 1 to 2, leader changed from 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66) to 69cd28922ac1481894b3a4e82609f71a (127.29.37.65). New cstate: current_term: 2 leader_uuid: "69cd28922ac1481894b3a4e82609f71a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:25.081117  8907 consensus_queue.cc:1048] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.081868  8933 consensus_queue.cc:1048] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:25.108752 29845 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:28:25.110730 29845 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:28:25.111142 29845 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:28:25.122537  8710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122"
dest_uuid: "69cd28922ac1481894b3a4e82609f71a"
mode: GRACEFUL
new_leader_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
 from {username='slave'} at 127.0.0.1:38796
I20260709 12:28:25.123044  8710 raft_consensus.cc:606] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 2 LEADER]: Received request to transfer leadership to TS d1ff59f123834cf9bf45ab6dcd5ff3eb
I20260709 12:28:25.124174 29845 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:28:25.125545  8858 raft_consensus.cc:1275] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Refusing update from remote peer 69cd28922ac1481894b3a4e82609f71a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:25.126968  8907 consensus_queue.cc:1048] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:25.136967  8784 raft_consensus.cc:1275] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Refusing update from remote peer 69cd28922ac1481894b3a4e82609f71a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:25.138505  8907 consensus_queue.cc:1048] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:25.361816  8858 raft_consensus.cc:1275] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Refusing update from remote peer 5aa9cc533d124a11b464147f0dc699ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:25.363473  8929 consensus_queue.cc:1048] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:25.378856  8710 raft_consensus.cc:1275] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Refusing update from remote peer 5aa9cc533d124a11b464147f0dc699ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:25.380946  8908 consensus_queue.cc:1048] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:25.483076  8784 raft_consensus.cc:1275] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Refusing update from remote peer 69cd28922ac1481894b3a4e82609f71a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:25.484436  8949 consensus_queue.cc:1048] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:25.495908  8858 raft_consensus.cc:1275] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Refusing update from remote peer 69cd28922ac1481894b3a4e82609f71a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:25.497689  8949 consensus_queue.cc:1048] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:25.505292  8907 raft_consensus.cc:993] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a: : Instructing follower d1ff59f123834cf9bf45ab6dcd5ff3eb to start an election
I20260709 12:28:25.505765  8949 raft_consensus.cc:1081] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 2 LEADER]: Signalling peer d1ff59f123834cf9bf45ab6dcd5ff3eb to start an election
I20260709 12:28:25.507521  8858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122"
dest_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb"
 from {username='slave'} at 127.0.0.1:49594
I20260709 12:28:25.508100  8858 raft_consensus.cc:493] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:25.508399  8858 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:25.516713  8858 raft_consensus.cc:515] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:25.519127  8858 leader_election.cc:290] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 3 election: Requested vote from peers 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985), 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303)
I20260709 12:28:25.520278  8710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "69cd28922ac1481894b3a4e82609f71a"
I20260709 12:28:25.520823  8784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4957601cc8e4de4a0c8c77dafb02122" candidate_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5aa9cc533d124a11b464147f0dc699ed"
I20260709 12:28:25.521068  8710 raft_consensus.cc:3055] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:25.521464  8784 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:25.521502  8710 raft_consensus.cc:740] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 69cd28922ac1481894b3a4e82609f71a, State: Running, Role: LEADER
I20260709 12:28:25.522781  8710 consensus_queue.cc:260] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:25.523816  8710 raft_consensus.cc:3060] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:25.530138  8784 raft_consensus.cc:2468] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 3.
I20260709 12:28:25.531327  8823 leader_election.cc:304] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa9cc533d124a11b464147f0dc699ed, d1ff59f123834cf9bf45ab6dcd5ff3eb; no voters: 
I20260709 12:28:25.531350  8710 raft_consensus.cc:2468] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1ff59f123834cf9bf45ab6dcd5ff3eb in term 3.
I20260709 12:28:25.532258  8912 raft_consensus.cc:2804] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:25.533274  8912 raft_consensus.cc:697] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 3 LEADER]: Becoming Leader. State: Replica: d1ff59f123834cf9bf45ab6dcd5ff3eb, State: Running, Role: LEADER
I20260709 12:28:25.534269  8912 consensus_queue.cc:237] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } }
I20260709 12:28:25.541620  8607 catalog_manager.cc:5697] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb reported cstate change: term changed from 2 to 3, leader changed from 69cd28922ac1481894b3a4e82609f71a (127.29.37.65) to d1ff59f123834cf9bf45ab6dcd5ff3eb (127.29.37.67). New cstate: current_term: 3 leader_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69cd28922ac1481894b3a4e82609f71a" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa9cc533d124a11b464147f0dc699ed" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 45303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1ff59f123834cf9bf45ab6dcd5ff3eb" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 45271 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:25.771443 29845 tablet_server.cc:179] TabletServer@127.29.37.65:0 shutting down...
W20260709 12:28:25.785132  8751 consensus_peers.cc:597] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed -> Peer 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985): Couldn't send request to peer 69cd28922ac1481894b3a4e82609f71a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:25.796851 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:25.797613 29845 tablet_replica.cc:333] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a: stopping tablet replica
I20260709 12:28:25.798326 29845 raft_consensus.cc:2243] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.798724 29845 raft_consensus.cc:2272] T b4957601cc8e4de4a0c8c77dafb02122 P 69cd28922ac1481894b3a4e82609f71a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.801692 29845 tablet_replica.cc:333] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a: stopping tablet replica
I20260709 12:28:25.802304 29845 raft_consensus.cc:2243] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:25.803185 29845 raft_consensus.cc:2272] T c7ff6e55aef24ff5a5060ae975b236e1 P 69cd28922ac1481894b3a4e82609f71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.806203 29845 tablet_replica.cc:333] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a: stopping tablet replica
I20260709 12:28:25.806799 29845 raft_consensus.cc:2243] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:25.807615 29845 raft_consensus.cc:2272] T a469562fdc7e44278ce957aafc5e7d7c P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.810462 29845 tablet_replica.cc:333] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a: stopping tablet replica
I20260709 12:28:25.810941 29845 raft_consensus.cc:2243] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.811360 29845 raft_consensus.cc:2272] T 294b9a834d2547348bc161068bbc3891 P 69cd28922ac1481894b3a4e82609f71a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.827839 29845 tablet_server.cc:196] TabletServer@127.29.37.65:0 shutdown complete.
I20260709 12:28:25.846297 29845 tablet_server.cc:179] TabletServer@127.29.37.66:0 shutting down...
I20260709 12:28:25.868548 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:25.869309 29845 tablet_replica.cc:333] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed: stopping tablet replica
I20260709 12:28:25.870111 29845 raft_consensus.cc:2243] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.870711 29845 raft_consensus.cc:2272] T c7ff6e55aef24ff5a5060ae975b236e1 P 5aa9cc533d124a11b464147f0dc699ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.873778 29845 tablet_replica.cc:333] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed: stopping tablet replica
I20260709 12:28:25.874414 29845 raft_consensus.cc:2243] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.874883 29845 raft_consensus.cc:2272] T b4957601cc8e4de4a0c8c77dafb02122 P 5aa9cc533d124a11b464147f0dc699ed [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.877341 29845 tablet_replica.cc:333] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed: stopping tablet replica
I20260709 12:28:25.877868 29845 raft_consensus.cc:2243] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.878446 29845 raft_consensus.cc:2272] T a469562fdc7e44278ce957aafc5e7d7c P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.881433 29845 tablet_replica.cc:333] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed: stopping tablet replica
I20260709 12:28:25.881942 29845 raft_consensus.cc:2243] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:25.882833 29845 raft_consensus.cc:2272] T 294b9a834d2547348bc161068bbc3891 P 5aa9cc533d124a11b464147f0dc699ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.908442 29845 tablet_server.cc:196] TabletServer@127.29.37.66:0 shutdown complete.
I20260709 12:28:25.927605 29845 tablet_server.cc:179] TabletServer@127.29.37.67:0 shutting down...
W20260709 12:28:25.940788  8823 consensus_peers.cc:597] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb -> Peer 5aa9cc533d124a11b464147f0dc699ed (127.29.37.66:45303): Couldn't send request to peer 5aa9cc533d124a11b464147f0dc699ed. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:45303: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:25.944706  8825 consensus_peers.cc:597] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb -> Peer 69cd28922ac1481894b3a4e82609f71a (127.29.37.65:35985): Couldn't send request to peer 69cd28922ac1481894b3a4e82609f71a. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:35985: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:25.955825 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:25.956509 29845 tablet_replica.cc:333] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb: stopping tablet replica
I20260709 12:28:25.957093 29845 raft_consensus.cc:2243] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:25.957803 29845 pending_rounds.cc:70] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Trying to abort 1 pending ops.
I20260709 12:28:25.957978 29845 pending_rounds.cc:77] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626032276934656 op_type: NO_OP noop_request { }
I20260709 12:28:25.958282 29845 raft_consensus.cc:2889] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:25.958557 29845 raft_consensus.cc:2272] T b4957601cc8e4de4a0c8c77dafb02122 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.961194 29845 tablet_replica.cc:333] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb: stopping tablet replica
I20260709 12:28:25.961683 29845 raft_consensus.cc:2243] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.962087 29845 raft_consensus.cc:2272] T c7ff6e55aef24ff5a5060ae975b236e1 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.965073 29845 tablet_replica.cc:333] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb: stopping tablet replica
I20260709 12:28:25.965590 29845 raft_consensus.cc:2243] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.966004 29845 raft_consensus.cc:2272] T 294b9a834d2547348bc161068bbc3891 P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.968469 29845 tablet_replica.cc:333] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb: stopping tablet replica
I20260709 12:28:25.968909 29845 raft_consensus.cc:2243] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:25.969419 29845 raft_consensus.cc:2272] T a469562fdc7e44278ce957aafc5e7d7c P d1ff59f123834cf9bf45ab6dcd5ff3eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:25.993808 29845 tablet_server.cc:196] TabletServer@127.29.37.67:0 shutdown complete.
I20260709 12:28:26.010646 29845 master.cc:562] Master@127.29.37.126:36177 shutting down...
I20260709 12:28:26.030491 29845 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:26.030994 29845 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:26.031291 29845 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3ba9fd274fa242a5acc209b4a5d363df: stopping tablet replica
I20260709 12:28:26.050506 29845 master.cc:584] Master@127.29.37.126:36177 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2788 ms)
[----------] 6 tests from LeaderRebalancerTest (371838 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:28:26.092908 29845 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.29.37.126:33431
I20260709 12:28:26.094269 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:26.099889  8963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:26.100369  8964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:26.101214  8966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:26.102275 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:26.103018 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:26.103193 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:26.103297 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600106103287 us; error 0 us; skew 500 ppm
I20260709 12:28:26.103751 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:26.105935 29845 webserver.cc:533] Webserver started at http://127.29.37.126:37947/ using document root <none> and password file <none>
I20260709 12:28:26.106428 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:26.106582 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:26.106802 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:26.107821 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/master-0-root/instance:
uuid: "6c56aae1034141c78d68463d8f5a3731"
format_stamp: "Formatted at 2026-07-09 12:28:26 on dist-test-slave-g1m6"
I20260709 12:28:26.112027 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 12:28:26.115185  8971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.115912 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:28:26.116139 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/master-0-root
uuid: "6c56aae1034141c78d68463d8f5a3731"
format_stamp: "Formatted at 2026-07-09 12:28:26 on dist-test-slave-g1m6"
I20260709 12:28:26.116364 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:26.133955 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:26.135083 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:26.172329 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.126:33431
I20260709 12:28:26.172448  9022 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.126:33431 every 8 connection(s)
I20260709 12:28:26.176213  9023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.187879  9023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731: Bootstrap starting.
I20260709 12:28:26.192792  9023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.196861  9023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731: No bootstrap required, opened a new log
I20260709 12:28:26.199054  9023 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c56aae1034141c78d68463d8f5a3731" member_type: VOTER }
I20260709 12:28:26.199436  9023 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.199674  9023 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c56aae1034141c78d68463d8f5a3731, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.200253  9023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c56aae1034141c78d68463d8f5a3731" member_type: VOTER }
I20260709 12:28:26.200726  9023 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:26.200958  9023 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:26.201202  9023 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:26.205781  9023 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c56aae1034141c78d68463d8f5a3731" member_type: VOTER }
I20260709 12:28:26.206317  9023 leader_election.cc:304] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c56aae1034141c78d68463d8f5a3731; no voters: 
I20260709 12:28:26.207398  9023 leader_election.cc:290] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:26.207800  9026 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:26.209117  9026 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 1 LEADER]: Becoming Leader. State: Replica: 6c56aae1034141c78d68463d8f5a3731, State: Running, Role: LEADER
I20260709 12:28:26.209794  9026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c56aae1034141c78d68463d8f5a3731" member_type: VOTER }
I20260709 12:28:26.210312  9023 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:26.212692  9028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c56aae1034141c78d68463d8f5a3731. Latest consensus state: current_term: 1 leader_uuid: "6c56aae1034141c78d68463d8f5a3731" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c56aae1034141c78d68463d8f5a3731" member_type: VOTER } }
I20260709 12:28:26.212702  9027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c56aae1034141c78d68463d8f5a3731" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c56aae1034141c78d68463d8f5a3731" member_type: VOTER } }
I20260709 12:28:26.213474  9028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:26.213551  9027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:26.216892  9031 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:26.224974  9031 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:26.225940 29845 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:26.233369  9031 catalog_manager.cc:1383] Generated new cluster ID: 8de642310f994e8e9a812175befcd0ef
I20260709 12:28:26.233639  9031 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:26.263725  9031 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:26.265110  9031 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:26.281409  9031 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731: Generated new TSK 0
I20260709 12:28:26.282004  9031 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:26.292454 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:26.298319  9044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:26.299474  9045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:26.300745  9047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:26.303068 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:26.304472 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:26.304679 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:26.304833 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600106304819 us; error 0 us; skew 500 ppm
I20260709 12:28:26.305411 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:26.307704 29845 webserver.cc:533] Webserver started at http://127.29.37.65:45149/ using document root <none> and password file <none>
I20260709 12:28:26.308175 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:26.308368 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:26.308626 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:26.309741 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-0-root/instance:
uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
format_stamp: "Formatted at 2026-07-09 12:28:26 on dist-test-slave-g1m6"
I20260709 12:28:26.314342 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:28:26.317451  9052 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.318255 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:26.318540 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
format_stamp: "Formatted at 2026-07-09 12:28:26 on dist-test-slave-g1m6"
I20260709 12:28:26.318840 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:26.334041 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:26.335158 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:26.336520 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:26.338806 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:26.339020 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.339262 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:26.339443 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.378679 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:38619
I20260709 12:28:26.378770  9114 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:38619 every 8 connection(s)
I20260709 12:28:26.383472 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:26.391309  9119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:26.391700  9120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:26.396363 29845 server_base.cc:1061] running on GCE node
W20260709 12:28:26.396708  9122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:26.397892  9115 heartbeater.cc:344] Connected to a master server at 127.29.37.126:33431
I20260709 12:28:26.398367  9115 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:26.398514 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:26.398813 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:26.398986 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600106398954 us; error 0 us; skew 500 ppm
I20260709 12:28:26.399224  9115 heartbeater.cc:507] Master 127.29.37.126:33431 requested a full tablet report, sending...
I20260709 12:28:26.399644 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:26.401590  8988 ts_manager.cc:194] Registered new tserver with Master: 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619)
I20260709 12:28:26.402505 29845 webserver.cc:533] Webserver started at http://127.29.37.66:42975/ using document root <none> and password file <none>
I20260709 12:28:26.403221 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:26.403461 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:26.403759 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:26.404179  8988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37698
I20260709 12:28:26.405439 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-1-root/instance:
uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
format_stamp: "Formatted at 2026-07-09 12:28:26 on dist-test-slave-g1m6"
I20260709 12:28:26.409830 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 12:28:26.413197  9127 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.413968 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:28:26.414352 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-1-root
uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
format_stamp: "Formatted at 2026-07-09 12:28:26 on dist-test-slave-g1m6"
I20260709 12:28:26.414654 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:26.431401 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:26.432597 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:26.434003 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:26.436259 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:26.436481 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.436718 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:26.436885 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.475878 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.66:44727
I20260709 12:28:26.475984  9189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.66:44727 every 8 connection(s)
I20260709 12:28:26.480907 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:26.488431  9193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:26.489514  9194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:26.492229  9196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:26.493276 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:26.493299  9190 heartbeater.cc:344] Connected to a master server at 127.29.37.126:33431
I20260709 12:28:26.493911  9190 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:26.494367 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:26.494624 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:26.494791  9190 heartbeater.cc:507] Master 127.29.37.126:33431 requested a full tablet report, sending...
I20260709 12:28:26.494825 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600106494790 us; error 0 us; skew 500 ppm
I20260709 12:28:26.495680 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:26.497218  8988 ts_manager.cc:194] Registered new tserver with Master: 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:26.498375 29845 webserver.cc:533] Webserver started at http://127.29.37.67:36989/ using document root <none> and password file <none>
I20260709 12:28:26.499043 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:26.499131  8988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37702
I20260709 12:28:26.499315 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:26.499724 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:26.501410 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-2-root/instance:
uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
format_stamp: "Formatted at 2026-07-09 12:28:26 on dist-test-slave-g1m6"
I20260709 12:28:26.505839 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:28:26.509069  9201 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.509814 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:28:26.510113 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-2-root
uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
format_stamp: "Formatted at 2026-07-09 12:28:26 on dist-test-slave-g1m6"
I20260709 12:28:26.510449 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599734091681-29845-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:26.530957 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:26.532083 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:26.533588 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:26.535733 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:26.535956 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.536198 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:26.536375 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:26.575017 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.67:46713
I20260709 12:28:26.575116  9263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.67:46713 every 8 connection(s)
I20260709 12:28:26.595974  9264 heartbeater.cc:344] Connected to a master server at 127.29.37.126:33431
I20260709 12:28:26.596494  9264 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:26.597642  9264 heartbeater.cc:507] Master 127.29.37.126:33431 requested a full tablet report, sending...
I20260709 12:28:26.600241  8988 ts_manager.cc:194] Registered new tserver with Master: 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:26.602205  8988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37718
I20260709 12:28:26.603446 29845 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.025191099s
I20260709 12:28:26.632843  8988 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37728:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:26.635540  8988 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:26.713774  9227 tablet_service.cc:1511] Processing CreateTablet for tablet 12f2b301f43d4850acaee49d3df55afa (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:26.714207  9226 tablet_service.cc:1511] Processing CreateTablet for tablet 2a77ba1e532f485abf98c54769535556 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:26.715389  9227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12f2b301f43d4850acaee49d3df55afa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.716521  9225 tablet_service.cc:1511] Processing CreateTablet for tablet 20ecb85c443e42c197335b7b69b97eff (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:26.717980  9225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ecb85c443e42c197335b7b69b97eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.721480  9229 tablet_service.cc:1511] Processing CreateTablet for tablet e2920b7e69b24a33bde09c66cd1a8db9 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:26.742635  9148 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1ca365e1da48cebf49533db981e646 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:26.743119  9229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2920b7e69b24a33bde09c66cd1a8db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.742842  9221 tablet_service.cc:1511] Processing CreateTablet for tablet 04120eea993a4a4c948f77e924c1cd39 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:26.743978  9148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1ca365e1da48cebf49533db981e646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.744616  9221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04120eea993a4a4c948f77e924c1cd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.723312  9154 tablet_service.cc:1511] Processing CreateTablet for tablet 68339bccb9ab48bf8a795f846dd75247 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:26.746469  9154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68339bccb9ab48bf8a795f846dd75247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.724985  9153 tablet_service.cc:1511] Processing CreateTablet for tablet 12f2b301f43d4850acaee49d3df55afa (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:26.751057  9153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12f2b301f43d4850acaee49d3df55afa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.751956  9147 tablet_service.cc:1511] Processing CreateTablet for tablet 04120eea993a4a4c948f77e924c1cd39 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:26.753288  9147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04120eea993a4a4c948f77e924c1cd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.726717  9224 tablet_service.cc:1511] Processing CreateTablet for tablet 92ad52027db5443b9e53187cfcbbf9e3 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:26.755146  9224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ad52027db5443b9e53187cfcbbf9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.726801  9152 tablet_service.cc:1511] Processing CreateTablet for tablet 2a77ba1e532f485abf98c54769535556 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:26.759049  9152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a77ba1e532f485abf98c54769535556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.728461  9150 tablet_service.cc:1511] Processing CreateTablet for tablet 92ad52027db5443b9e53187cfcbbf9e3 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:26.763178  9150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ad52027db5443b9e53187cfcbbf9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.728991  9149 tablet_service.cc:1511] Processing CreateTablet for tablet 0c01405fa79f45278781c477b429c667 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:26.767000  9149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c01405fa79f45278781c477b429c667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.730271  9151 tablet_service.cc:1511] Processing CreateTablet for tablet 20ecb85c443e42c197335b7b69b97eff (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:26.771013  9151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ecb85c443e42c197335b7b69b97eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.730822  9223 tablet_service.cc:1511] Processing CreateTablet for tablet 0c01405fa79f45278781c477b429c667 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:26.732205  9228 tablet_service.cc:1511] Processing CreateTablet for tablet 68339bccb9ab48bf8a795f846dd75247 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:26.735312  9079 tablet_service.cc:1511] Processing CreateTablet for tablet 68339bccb9ab48bf8a795f846dd75247 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:26.775066  9223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c01405fa79f45278781c477b429c667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.775681  9079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68339bccb9ab48bf8a795f846dd75247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.778836  9228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68339bccb9ab48bf8a795f846dd75247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.737051  9078 tablet_service.cc:1511] Processing CreateTablet for tablet 12f2b301f43d4850acaee49d3df55afa (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:26.782573  9226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a77ba1e532f485abf98c54769535556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.783270  9078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12f2b301f43d4850acaee49d3df55afa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.737246  9222 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1ca365e1da48cebf49533db981e646 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:26.738890  9077 tablet_service.cc:1511] Processing CreateTablet for tablet 2a77ba1e532f485abf98c54769535556 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:26.739693  9076 tablet_service.cc:1511] Processing CreateTablet for tablet 20ecb85c443e42c197335b7b69b97eff (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:26.740517  9080 tablet_service.cc:1511] Processing CreateTablet for tablet e2920b7e69b24a33bde09c66cd1a8db9 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:26.741204  9155 tablet_service.cc:1511] Processing CreateTablet for tablet e2920b7e69b24a33bde09c66cd1a8db9 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:26.791818  9222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1ca365e1da48cebf49533db981e646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.792059  9281 tablet_bootstrap.cc:492] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:26.795580  9075 tablet_service.cc:1511] Processing CreateTablet for tablet 92ad52027db5443b9e53187cfcbbf9e3 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:26.797746  9075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ad52027db5443b9e53187cfcbbf9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.806800  9077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a77ba1e532f485abf98c54769535556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.810405  9155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2920b7e69b24a33bde09c66cd1a8db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.819278  9073 tablet_service.cc:1511] Processing CreateTablet for tablet 7f1ca365e1da48cebf49533db981e646 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:26.816709  9074 tablet_service.cc:1511] Processing CreateTablet for tablet 0c01405fa79f45278781c477b429c667 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:26.820686  9073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f1ca365e1da48cebf49533db981e646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.821269  9074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c01405fa79f45278781c477b429c667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.826821  9072 tablet_service.cc:1511] Processing CreateTablet for tablet 04120eea993a4a4c948f77e924c1cd39 (DEFAULT_TABLE table=test-workload [id=5bf94d719a8c40ccb5f474b252324574]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:26.828166  9072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04120eea993a4a4c948f77e924c1cd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.849954  9076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ecb85c443e42c197335b7b69b97eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.851006  9080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2920b7e69b24a33bde09c66cd1a8db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:26.875851  9281 tablet_bootstrap.cc:654] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.887290  9280 tablet_bootstrap.cc:492] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:26.899613  9282 tablet_bootstrap.cc:492] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:26.908172  9282 tablet_bootstrap.cc:654] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.945168  9280 tablet_bootstrap.cc:654] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.946302  9281 tablet_bootstrap.cc:492] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:26.946841  9281 ts_tablet_manager.cc:1403] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.000s
I20260709 12:28:26.949903  9281 raft_consensus.cc:359] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:26.950667  9281 raft_consensus.cc:385] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.950955  9281 raft_consensus.cc:740] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.951733  9281 consensus_queue.cc:260] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:26.954268  9190 heartbeater.cc:499] Master 127.29.37.126:33431 was elected leader, sending a full tablet report...
I20260709 12:28:26.963541  9281 ts_tablet_manager.cc:1434] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:28:26.964598  9281 tablet_bootstrap.cc:492] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:26.965845  9282 tablet_bootstrap.cc:492] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:26.966395  9282 ts_tablet_manager.cc:1403] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.067s	user 0.020s	sys 0.023s
I20260709 12:28:26.969512  9282 raft_consensus.cc:359] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:26.971052  9281 tablet_bootstrap.cc:654] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:26.978513  9282 raft_consensus.cc:385] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.979089  9282 raft_consensus.cc:740] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.980288  9282 consensus_queue.cc:260] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:26.985185  9286 raft_consensus.cc:493] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.985404  9280 tablet_bootstrap.cc:492] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:26.985981  9280 ts_tablet_manager.cc:1403] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.099s	user 0.024s	sys 0.024s
I20260709 12:28:26.985728  9286 raft_consensus.cc:515] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:26.989173  9280 raft_consensus.cc:359] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:26.989897  9280 raft_consensus.cc:385] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:26.990195  9280 raft_consensus.cc:740] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:26.991016  9280 consensus_queue.cc:260] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:26.994658  9286 leader_election.cc:290] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:26.997550  9281 tablet_bootstrap.cc:492] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:26.998416  9281 ts_tablet_manager.cc:1403] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.009s
I20260709 12:28:27.002200  9281 raft_consensus.cc:359] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.004350  9115 heartbeater.cc:499] Master 127.29.37.126:33431 was elected leader, sending a full tablet report...
I20260709 12:28:27.005523  9264 heartbeater.cc:499] Master 127.29.37.126:33431 was elected leader, sending a full tablet report...
I20260709 12:28:27.006101  9280 ts_tablet_manager.cc:1434] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.012s
I20260709 12:28:27.007156  9280 tablet_bootstrap.cc:492] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:27.004374  9281 raft_consensus.cc:385] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.009905  9281 raft_consensus.cc:740] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.010877  9281 consensus_queue.cc:260] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.012861  9282 ts_tablet_manager.cc:1434] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.046s	user 0.019s	sys 0.006s
I20260709 12:28:27.014056  9280 tablet_bootstrap.cc:654] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.015043  9282 tablet_bootstrap.cc:492] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:27.018124  9281 ts_tablet_manager.cc:1434] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:28:27.019424  9281 tablet_bootstrap.cc:492] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:27.026278  9282 tablet_bootstrap.cc:654] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.026904  9288 raft_consensus.cc:493] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.029845  9288 raft_consensus.cc:515] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.033849  9281 tablet_bootstrap.cc:654] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.037164  9288 leader_election.cc:290] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.043743  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:27.044684  9090 raft_consensus.cc:2468] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:27.047171  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.049826  9282 tablet_bootstrap.cc:492] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:27.050130  9280 tablet_bootstrap.cc:492] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:27.050379  9282 ts_tablet_manager.cc:1403] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.007s
I20260709 12:28:27.050844  9280 ts_tablet_manager.cc:1403] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.024s
I20260709 12:28:27.053576  9282 raft_consensus.cc:359] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.054347  9282 raft_consensus.cc:385] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.054651  9282 raft_consensus.cc:740] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.055431  9282 consensus_queue.cc:260] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.055819  9280 raft_consensus.cc:359] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.056501  9128 leader_election.cc:304] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.056849  9280 raft_consensus.cc:385] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.057288  9280 raft_consensus.cc:740] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.058516  9281 tablet_bootstrap.cc:492] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:27.059211  9281 ts_tablet_manager.cc:1403] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.005s
I20260709 12:28:27.058462  9280 consensus_queue.cc:260] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.061795  9280 ts_tablet_manager.cc:1434] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 12:28:27.062590  9282 ts_tablet_manager.cc:1434] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:28:27.063030  9280 tablet_bootstrap.cc:492] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:27.063122  9286 raft_consensus.cc:2804] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.063741  9286 raft_consensus.cc:493] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.064083  9286 raft_consensus.cc:3060] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.064276  9281 raft_consensus.cc:359] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.065030  9281 raft_consensus.cc:385] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.065315  9281 raft_consensus.cc:740] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.066104  9281 consensus_queue.cc:260] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
W20260709 12:28:27.067474  9128 leader_election.cc:343] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.067818  9282 tablet_bootstrap.cc:492] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:27.076507  9286 raft_consensus.cc:515] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.078488  9280 tablet_bootstrap.cc:654] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.079890  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:27.080682  9090 raft_consensus.cc:3060] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.082161  9282 tablet_bootstrap.cc:654] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.093900  9090 raft_consensus.cc:2468] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:27.095520  9128 leader_election.cc:304] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.098778  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
W20260709 12:28:27.100194  9128 leader_election.cc:343] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.100922  9299 raft_consensus.cc:2804] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.101387  9286 leader_election.cc:290] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.102608  9299 raft_consensus.cc:697] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:27.103930  9299 consensus_queue.cc:237] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.107470  9282 tablet_bootstrap.cc:492] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:27.108186  9282 ts_tablet_manager.cc:1403] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.002s
I20260709 12:28:27.111711  9282 raft_consensus.cc:359] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.112874  9282 raft_consensus.cc:385] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.113238  9282 raft_consensus.cc:740] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.116288  9282 consensus_queue.cc:260] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.117301  9280 tablet_bootstrap.cc:492] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:27.118160  9280 ts_tablet_manager.cc:1403] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.055s	user 0.018s	sys 0.016s
I20260709 12:28:27.118881  9282 ts_tablet_manager.cc:1434] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 12:28:27.118973  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:27.120102  9282 tablet_bootstrap.cc:492] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:27.123306  9281 ts_tablet_manager.cc:1434] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.064s	user 0.010s	sys 0.007s
I20260709 12:28:27.124603  9281 tablet_bootstrap.cc:492] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:27.121277  9280 raft_consensus.cc:359] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.131107  9280 raft_consensus.cc:385] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.131412  9280 raft_consensus.cc:740] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.132216  9280 consensus_queue.cc:260] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.132416  9282 tablet_bootstrap.cc:654] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.134315  9280 ts_tablet_manager.cc:1434] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:28:27.135339  9280 tablet_bootstrap.cc:492] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:27.144918  9282 tablet_bootstrap.cc:492] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:27.145792  9282 ts_tablet_manager.cc:1403] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.009s
I20260709 12:28:27.147971  9281 tablet_bootstrap.cc:654] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.148401  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
W20260709 12:28:27.150077  9053 leader_election.cc:343] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.149622  9282 raft_consensus.cc:359] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.150599  9053 leader_election.cc:304] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:27.150693  9282 raft_consensus.cc:385] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.151270  9282 raft_consensus.cc:740] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.151875  9288 raft_consensus.cc:2749] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.152148  9282 consensus_queue.cc:260] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.154511  9282 ts_tablet_manager.cc:1434] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:28:27.155500  9282 tablet_bootstrap.cc:492] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:27.156742  8988 catalog_manager.cc:5697] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 1 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.158043  9280 tablet_bootstrap.cc:654] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.161887  9282 tablet_bootstrap.cc:654] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.167284  9281 tablet_bootstrap.cc:492] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:27.167925  9281 ts_tablet_manager.cc:1403] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.044s	user 0.004s	sys 0.010s
I20260709 12:28:27.168607  9288 raft_consensus.cc:493] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.169286  9288 raft_consensus.cc:515] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.171664  9281 raft_consensus.cc:359] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.172451  9281 raft_consensus.cc:385] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.172734  9281 raft_consensus.cc:740] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.174248  9282 tablet_bootstrap.cc:492] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:27.174759  9282 ts_tablet_manager.cc:1403] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.011s
I20260709 12:28:27.174546  9281 consensus_queue.cc:260] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.175789  9280 tablet_bootstrap.cc:492] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:27.176307  9280 ts_tablet_manager.cc:1403] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.009s
I20260709 12:28:27.177280  9281 ts_tablet_manager.cc:1434] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:28:27.177883  9282 raft_consensus.cc:359] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.178349  9281 tablet_bootstrap.cc:492] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:27.178647  9282 raft_consensus.cc:385] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.178968  9282 raft_consensus.cc:740] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.179407  9280 raft_consensus.cc:359] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.180146  9280 raft_consensus.cc:385] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.179812  9282 consensus_queue.cc:260] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.180465  9280 raft_consensus.cc:740] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.181254  9280 consensus_queue.cc:260] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.186923  9281 tablet_bootstrap.cc:654] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.187575  9288 leader_election.cc:290] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.189127  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68339bccb9ab48bf8a795f846dd75247" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:27.189419  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68339bccb9ab48bf8a795f846dd75247" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.189679  9282 ts_tablet_manager.cc:1434] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 12:28:27.190071  9239 raft_consensus.cc:2468] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
W20260709 12:28:27.190640  9053 leader_election.cc:343] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.190783  9282 tablet_bootstrap.cc:492] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:27.191438  9053 leader_election.cc:304] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:27.192229  9288 raft_consensus.cc:2804] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.192593  9288 raft_consensus.cc:493] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.192924  9288 raft_consensus.cc:3060] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.193548  9280 ts_tablet_manager.cc:1434] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 12:28:27.194662  9280 tablet_bootstrap.cc:492] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:27.198108  9282 tablet_bootstrap.cc:654] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.201326  9280 tablet_bootstrap.cc:654] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.201555  9288 raft_consensus.cc:515] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.205045  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68339bccb9ab48bf8a795f846dd75247" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
W20260709 12:28:27.206965  9053 leader_election.cc:343] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.209952  9282 tablet_bootstrap.cc:492] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:27.210493  9282 ts_tablet_manager.cc:1403] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.001s
I20260709 12:28:27.210949  9288 leader_election.cc:290] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.211505  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68339bccb9ab48bf8a795f846dd75247" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:27.212252  9288 raft_consensus.cc:493] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.212356  9239 raft_consensus.cc:3060] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.212749  9288 raft_consensus.cc:515] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.213886  9282 raft_consensus.cc:359] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.214732  9282 raft_consensus.cc:385] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.215037  9282 raft_consensus.cc:740] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.215673  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a77ba1e532f485abf98c54769535556" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
W20260709 12:28:27.217087  9053 leader_election.cc:343] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.216985  9282 consensus_queue.cc:260] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.218734  9280 tablet_bootstrap.cc:492] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:27.219336  9280 ts_tablet_manager.cc:1403] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.016s
W20260709 12:28:27.220693  9036 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:27.221954  9238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a77ba1e532f485abf98c54769535556" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.222465  9288 leader_election.cc:290] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.222750  9282 ts_tablet_manager.cc:1434] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
W20260709 12:28:27.223532  9053 leader_election.cc:343] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.224079  9053 leader_election.cc:304] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:27.225315  9280 raft_consensus.cc:359] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.224040  9282 tablet_bootstrap.cc:492] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:27.226153  9280 raft_consensus.cc:385] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.226913  9280 raft_consensus.cc:740] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.228130  9281 tablet_bootstrap.cc:492] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:27.228664  9281 ts_tablet_manager.cc:1403] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.011s
I20260709 12:28:27.227871  9280 consensus_queue.cc:260] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.231523  9280 ts_tablet_manager.cc:1434] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:28:27.232601  9280 tablet_bootstrap.cc:492] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:27.233009  9282 tablet_bootstrap.cc:654] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.231788  9281 raft_consensus.cc:359] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.234625  9281 raft_consensus.cc:385] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.234939  9281 raft_consensus.cc:740] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.226055  9288 raft_consensus.cc:2749] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.235852  9281 consensus_queue.cc:260] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.239454  9281 ts_tablet_manager.cc:1434] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:28:27.240833  9281 tablet_bootstrap.cc:492] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:27.243084  9299 raft_consensus.cc:493] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.243814  9299 raft_consensus.cc:515] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.247506  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ecb85c443e42c197335b7b69b97eff" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:27.248349  9090 raft_consensus.cc:2468] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:27.249986  9128 leader_election.cc:304] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.251195  9296 raft_consensus.cc:2804] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.251617  9296 raft_consensus.cc:493] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.251950  9296 raft_consensus.cc:3060] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.252408  9238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ecb85c443e42c197335b7b69b97eff" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
W20260709 12:28:27.253952  9128 leader_election.cc:343] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:27.256284  9280 tablet_bootstrap.cc:654] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.251215  9299 leader_election.cc:290] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.258545  9281 tablet_bootstrap.cc:654] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.261420  9296 raft_consensus.cc:515] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.268155  9280 tablet_bootstrap.cc:492] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:27.268755  9280 ts_tablet_manager.cc:1403] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.003s
I20260709 12:28:27.269119  9296 leader_election.cc:290] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.266033  9239 raft_consensus.cc:2468] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:27.269233  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ecb85c443e42c197335b7b69b97eff" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:27.270690  9090 raft_consensus.cc:3060] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.271304  9053 leader_election.cc:304] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:27.272485  9288 raft_consensus.cc:2804] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.272974  9288 raft_consensus.cc:697] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:27.269778  9238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ecb85c443e42c197335b7b69b97eff" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:27.273808  9288 consensus_queue.cc:237] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.272965  9282 tablet_bootstrap.cc:492] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:27.274117  9280 raft_consensus.cc:359] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.275102  9282 ts_tablet_manager.cc:1403] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.033s
I20260709 12:28:27.275151  9280 raft_consensus.cc:385] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.275707  9280 raft_consensus.cc:740] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.276710  9280 consensus_queue.cc:260] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.277633  9238 raft_consensus.cc:3060] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.278404  9282 raft_consensus.cc:359] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.278579  9090 raft_consensus.cc:2468] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:27.279191  9282 raft_consensus.cc:385] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.279548  9282 raft_consensus.cc:740] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.280285  9128 leader_election.cc:304] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.281272  9296 raft_consensus.cc:2804] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.281759  9296 raft_consensus.cc:697] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:27.280330  9282 consensus_queue.cc:260] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.282681  9296 consensus_queue.cc:237] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.283270  9281 tablet_bootstrap.cc:492] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:27.283779  9281 ts_tablet_manager.cc:1403] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.007s
I20260709 12:28:27.285097  9282 ts_tablet_manager.cc:1434] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:28:27.286533  9282 tablet_bootstrap.cc:492] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:27.286864  9281 raft_consensus.cc:359] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.287648  9281 raft_consensus.cc:385] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.287947  9281 raft_consensus.cc:740] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.288837  9281 consensus_queue.cc:260] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.289652  9299 raft_consensus.cc:493] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.290164  9299 raft_consensus.cc:515] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.291916  9286 raft_consensus.cc:493] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.292243  9299 leader_election.cc:290] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.292457  9286 raft_consensus.cc:515] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.294487  9286 leader_election.cc:290] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.294906  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04120eea993a4a4c948f77e924c1cd39" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:27.295333  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04120eea993a4a4c948f77e924c1cd39" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.295699  9090 raft_consensus.cc:2468] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:27.296043  9239 raft_consensus.cc:2468] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:27.297909  9128 leader_election.cc:304] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.298342  9282 tablet_bootstrap.cc:654] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.299747  9238 raft_consensus.cc:2468] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:27.300676  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68339bccb9ab48bf8a795f846dd75247" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.301584  9286 raft_consensus.cc:2804] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.301965  9239 raft_consensus.cc:2393] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97b41bf0005c4e9c8a9ed4795fc9080f in current term 1: Already voted for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in this term.
I20260709 12:28:27.300679  8985 catalog_manager.cc:5697] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.301972  9286 raft_consensus.cc:493] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.303730  9286 raft_consensus.cc:3060] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.313887  9280 ts_tablet_manager.cc:1434] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 12:28:27.317780  9280 tablet_bootstrap.cc:492] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:27.324656  9280 tablet_bootstrap.cc:654] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.302680  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68339bccb9ab48bf8a795f846dd75247" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:27.328317  8987 catalog_manager.cc:5697] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 1 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.330816  9282 tablet_bootstrap.cc:492] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:27.331337  9282 ts_tablet_manager.cc:1403] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.045s	user 0.022s	sys 0.018s
I20260709 12:28:27.334136  9282 raft_consensus.cc:359] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.334805  9282 raft_consensus.cc:385] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.335063  9282 raft_consensus.cc:740] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.335858  9282 consensus_queue.cc:260] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.291595  9281 ts_tablet_manager.cc:1434] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 12:28:27.339453  9281 tablet_bootstrap.cc:492] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:27.340893  9282 ts_tablet_manager.cc:1434] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:27.341881  9282 tablet_bootstrap.cc:492] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:27.341554  9128 leader_election.cc:304] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5
I20260709 12:28:27.342769  9296 raft_consensus.cc:3060] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.348237  9282 tablet_bootstrap.cc:654] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.349026  9286 raft_consensus.cc:515] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.351430  9286 leader_election.cc:290] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.357738  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04120eea993a4a4c948f77e924c1cd39" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:27.358472  9238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04120eea993a4a4c948f77e924c1cd39" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:27.358594  9090 raft_consensus.cc:3060] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.359103  9238 raft_consensus.cc:3060] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.360127  9296 raft_consensus.cc:2749] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.366873  9238 raft_consensus.cc:2468] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:27.368145  9128 leader_election.cc:304] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.368566  9288 raft_consensus.cc:493] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.369143  9286 raft_consensus.cc:2804] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.369076  9288 raft_consensus.cc:515] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.369199  9090 raft_consensus.cc:2468] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:27.371354  9288 leader_election.cc:290] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.372048  9286 raft_consensus.cc:697] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:27.372983  9286 consensus_queue.cc:237] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.373376  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ad52027db5443b9e53187cfcbbf9e3" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:27.374706  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ad52027db5443b9e53187cfcbbf9e3" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.375603  9239 raft_consensus.cc:2468] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:27.376597  9165 raft_consensus.cc:2468] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:27.376842  9053 leader_election.cc:304] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:27.378315  9288 raft_consensus.cc:2804] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.378638  9281 tablet_bootstrap.cc:654] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.378777  9288 raft_consensus.cc:493] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.379293  9288 raft_consensus.cc:3060] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.382884  8987 catalog_manager.cc:5697] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 1 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.384362  9280 tablet_bootstrap.cc:492] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:27.384898  9280 ts_tablet_manager.cc:1403] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.022s
I20260709 12:28:27.387753  9288 raft_consensus.cc:515] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.388036  9280 raft_consensus.cc:359] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.389315  9282 tablet_bootstrap.cc:492] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:27.389524  9280 raft_consensus.cc:385] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.389030  9281 tablet_bootstrap.cc:492] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:27.390004  9282 ts_tablet_manager.cc:1403] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.048s	user 0.028s	sys 0.008s
I20260709 12:28:27.390568  9281 ts_tablet_manager.cc:1403] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.001s
I20260709 12:28:27.389946  9280 raft_consensus.cc:740] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.394949  9282 raft_consensus.cc:359] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.394851  9280 consensus_queue.cc:260] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.395502  9282 raft_consensus.cc:385] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.395439  9281 raft_consensus.cc:359] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.391858  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ad52027db5443b9e53187cfcbbf9e3" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:27.396374  9281 raft_consensus.cc:385] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.396600  9239 raft_consensus.cc:3060] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.396759  9281 raft_consensus.cc:740] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.397315  9280 ts_tablet_manager.cc:1434] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:28:27.397666  9281 consensus_queue.cc:260] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.398488  9280 tablet_bootstrap.cc:492] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:27.392323  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ad52027db5443b9e53187cfcbbf9e3" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:27.402479  9165 raft_consensus.cc:3060] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.406036  9280 tablet_bootstrap.cc:654] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.407516  9281 ts_tablet_manager.cc:1434] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:28:27.411005  9165 raft_consensus.cc:2468] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:27.411761  9281 tablet_bootstrap.cc:492] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:27.396023  9282 raft_consensus.cc:740] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.390833  9288 leader_election.cc:290] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.414436  9239 raft_consensus.cc:2468] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:27.414536  9053 leader_election.cc:304] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.415455  9288 raft_consensus.cc:2804] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.415918  9288 raft_consensus.cc:697] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:27.415575  9282 consensus_queue.cc:260] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.416849  9288 consensus_queue.cc:237] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.417853  9282 ts_tablet_manager.cc:1434] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.000s
I20260709 12:28:27.418479  9281 tablet_bootstrap.cc:654] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.421926  9280 tablet_bootstrap.cc:492] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:27.422390  9280 ts_tablet_manager.cc:1403] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.000s
I20260709 12:28:27.425019  9280 raft_consensus.cc:359] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.425522  9280 raft_consensus.cc:385] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.425752  9280 raft_consensus.cc:740] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.426071  9281 tablet_bootstrap.cc:492] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:27.426397  9280 consensus_queue.cc:260] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.426595  9281 ts_tablet_manager.cc:1403] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.003s
I20260709 12:28:27.426507  8987 catalog_manager.cc:5697] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.428411  9280 ts_tablet_manager.cc:1434] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:27.429352  9280 tablet_bootstrap.cc:492] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:27.429569  9281 raft_consensus.cc:359] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.430282  9281 raft_consensus.cc:385] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.430538  9281 raft_consensus.cc:740] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.431322  9281 consensus_queue.cc:260] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.433224  9281 ts_tablet_manager.cc:1434] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:27.436079  9281 tablet_bootstrap.cc:492] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:27.436774  9280 tablet_bootstrap.cc:654] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.441869  9281 tablet_bootstrap.cc:654] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:27.447537  9280 tablet_bootstrap.cc:492] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:27.447942  9280 ts_tablet_manager.cc:1403] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:28:27.450342  9280 raft_consensus.cc:359] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.450992  9280 raft_consensus.cc:385] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.451241  9280 raft_consensus.cc:740] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.451884  9280 consensus_queue.cc:260] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.453873  9280 ts_tablet_manager.cc:1434] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:27.457001  9281 tablet_bootstrap.cc:492] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:27.457532  9281 ts_tablet_manager.cc:1403] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 12:28:27.460641  9281 raft_consensus.cc:359] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.461251  9281 raft_consensus.cc:385] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:27.461516  9281 raft_consensus.cc:740] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:27.462106  9281 consensus_queue.cc:260] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.464030  9281 ts_tablet_manager.cc:1434] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:27.468786  9289 raft_consensus.cc:493] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.469379  9289 raft_consensus.cc:515] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.470024  9288 raft_consensus.cc:493] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.470634  9288 raft_consensus.cc:515] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.471884  9289 leader_election.cc:290] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:27.472890  9288 leader_election.cc:290] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.475028  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:27.475766  9165 raft_consensus.cc:2468] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:27.477346  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.477627  9053 leader_election.cc:304] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.478029  9239 raft_consensus.cc:2468] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:27.478565  9288 raft_consensus.cc:2804] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.478946  9288 raft_consensus.cc:493] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.479313  9288 raft_consensus.cc:3060] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.482012  9305 raft_consensus.cc:493] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.482592  9305 raft_consensus.cc:515] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.483525  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:27.484898  9305 leader_election.cc:290] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.487679  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:27.488363  9165 raft_consensus.cc:2468] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 0.
I20260709 12:28:27.489657  9202 leader_election.cc:304] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.489928  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c01405fa79f45278781c477b429c667" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.489943  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c01405fa79f45278781c477b429c667" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:27.490720  9239 raft_consensus.cc:2468] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:27.490787  9289 raft_consensus.cc:2804] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.490839  9165 raft_consensus.cc:2468] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:27.489527  9288 raft_consensus.cc:515] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.491452  9289 raft_consensus.cc:493] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.491912  9289 raft_consensus.cc:3060] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.492198  9090 raft_consensus.cc:2393] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 196d3384261d4a8dbf0a91f553b1ef4a in current term 1: Already voted for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in this term.
I20260709 12:28:27.492304  9053 leader_election.cc:304] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.493263  9305 raft_consensus.cc:2804] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.493667  9305 raft_consensus.cc:493] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.494010  9305 raft_consensus.cc:3060] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.496438  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:27.497031  9165 raft_consensus.cc:3060] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.497779  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:27.499121  9288 leader_election.cc:290] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.500129  9289 raft_consensus.cc:515] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.501233  9239 raft_consensus.cc:2393] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f93ba6f3ac741ee93fa5765a94326e5 in current term 1: Already voted for candidate 196d3384261d4a8dbf0a91f553b1ef4a in this term.
I20260709 12:28:27.501040  9305 raft_consensus.cc:515] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.503088  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:27.503309  9305 leader_election.cc:290] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.503715  9289 leader_election.cc:290] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:27.503880  9090 raft_consensus.cc:2393] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 196d3384261d4a8dbf0a91f553b1ef4a in current term 1: Already voted for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in this term.
I20260709 12:28:27.505048  9164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:27.505462  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c01405fa79f45278781c477b429c667" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:27.506120  9163 raft_consensus.cc:3060] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.506764  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c01405fa79f45278781c477b429c667" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:27.506826  9202 leader_election.cc:304] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:27.507496  9239 raft_consensus.cc:3060] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.510421  9289 raft_consensus.cc:2749] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:27.511718  9163 raft_consensus.cc:2468] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:27.506109  9165 raft_consensus.cc:2468] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:27.512809  9053 leader_election.cc:304] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.513293  9239 raft_consensus.cc:2468] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:27.513569  9305 raft_consensus.cc:2804] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.513759  9053 leader_election.cc:304] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:27.514271  9305 raft_consensus.cc:697] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:27.515214  9288 raft_consensus.cc:2804] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.515251  9305 consensus_queue.cc:237] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.515970  9288 raft_consensus.cc:697] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:27.516826  9288 consensus_queue.cc:237] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.523797  8987 catalog_manager.cc:5697] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.535584  8987 catalog_manager.cc:5697] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.537535  9289 raft_consensus.cc:493] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.538020  9289 raft_consensus.cc:515] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.540778  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2920b7e69b24a33bde09c66cd1a8db9" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:27.540217  9289 leader_election.cc:290] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:27.541136  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2920b7e69b24a33bde09c66cd1a8db9" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:27.541489  9090 raft_consensus.cc:2468] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 0.
I20260709 12:28:27.541882  9165 raft_consensus.cc:2468] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 0.
I20260709 12:28:27.542635  9202 leader_election.cc:304] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:27.543444  9289 raft_consensus.cc:2804] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.543787  9289 raft_consensus.cc:493] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.544113  9289 raft_consensus.cc:3060] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.549289  9289 raft_consensus.cc:515] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.550835  9289 leader_election.cc:290] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:27.551558  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2920b7e69b24a33bde09c66cd1a8db9" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:27.551682  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2920b7e69b24a33bde09c66cd1a8db9" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:27.552091  9090 raft_consensus.cc:3060] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.552222  9165 raft_consensus.cc:3060] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.556962  9090 raft_consensus.cc:2468] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 1.
I20260709 12:28:27.557165  9165 raft_consensus.cc:2468] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 1.
I20260709 12:28:27.558017  9202 leader_election.cc:304] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:27.558733  9289 raft_consensus.cc:2804] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.559535  9289 raft_consensus.cc:697] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:27.560180  9289 consensus_queue.cc:237] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:27.567418  8987 catalog_manager.cc:5697] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 1 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.587548  9239 raft_consensus.cc:3060] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.594008  9286 consensus_queue.cc:1048] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.594959  9296 consensus_queue.cc:1048] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.673875  9286 raft_consensus.cc:493] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.674431  9286 raft_consensus.cc:515] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.676234  9286 leader_election.cc:290] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.677151  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a77ba1e532f485abf98c54769535556" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:27.677318  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a77ba1e532f485abf98c54769535556" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:27.677760  9090 raft_consensus.cc:2468] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:27.677977  9239 raft_consensus.cc:2468] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:27.678831  9128 leader_election.cc:304] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.679498  9286 raft_consensus.cc:2804] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:27.679817  9286 raft_consensus.cc:493] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.680086  9286 raft_consensus.cc:3060] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.685312  9286 raft_consensus.cc:515] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.687211  9286 leader_election.cc:290] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.687805  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a77ba1e532f485abf98c54769535556" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:27.687644  9296 consensus_queue.cc:1048] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.688409  9090 raft_consensus.cc:3060] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.688513  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a77ba1e532f485abf98c54769535556" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:27.689119  9239 raft_consensus.cc:3060] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:27.695734  9090 raft_consensus.cc:2468] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:27.695948  9239 raft_consensus.cc:2468] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:27.697827  9128 leader_election.cc:304] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.698626  9296 raft_consensus.cc:2804] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:27.699079  9296 raft_consensus.cc:697] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:27.699940  9296 consensus_queue.cc:237] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.704698  9337 consensus_queue.cc:1048] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.718918  8987 catalog_manager.cc:5697] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 1 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.732796  9305 consensus_queue.cc:1048] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.736014  9305 consensus_queue.cc:1048] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.770356  9296 consensus_queue.cc:1048] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.780992  9286 consensus_queue.cc:1048] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.861208  9239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2920b7e69b24a33bde09c66cd1a8db9"
dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
mode: GRACEFUL
new_leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
 from {username='slave'} at 127.0.0.1:39010
I20260709 12:28:27.861908  9239 raft_consensus.cc:606] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Received request to transfer leadership to TS 3f93ba6f3ac741ee93fa5765a94326e5
I20260709 12:28:27.882743  9165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a77ba1e532f485abf98c54769535556"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
mode: GRACEFUL
new_leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
 from {username='slave'} at 127.0.0.1:58214
I20260709 12:28:27.883307  9165 raft_consensus.cc:606] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Received request to transfer leadership to TS 3f93ba6f3ac741ee93fa5765a94326e5
I20260709 12:28:27.893754  9288 consensus_queue.cc:1048] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.900848  9090 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92ad52027db5443b9e53187cfcbbf9e3"
dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
mode: GRACEFUL
new_leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:34110
I20260709 12:28:27.901449  9090 raft_consensus.cc:606] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Received request to transfer leadership to TS 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:27.905263  9305 consensus_queue.cc:1048] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.907603  9090 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c01405fa79f45278781c477b429c667"
dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
mode: GRACEFUL
new_leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:34110
I20260709 12:28:27.908205  9090 raft_consensus.cc:606] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Received request to transfer leadership to TS 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:27.921569  9288 raft_consensus.cc:993] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5: : Instructing follower 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:27.924551  9305 raft_consensus.cc:1081] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Signalling peer 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:27.925117  9165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04120eea993a4a4c948f77e924c1cd39"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
mode: GRACEFUL
new_leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:58214
I20260709 12:28:27.925810  9165 raft_consensus.cc:606] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Received request to transfer leadership to TS 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:27.926427  9163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92ad52027db5443b9e53187cfcbbf9e3"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:58188
I20260709 12:28:27.927006  9163 raft_consensus.cc:493] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:27.927379  9163 raft_consensus.cc:3060] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:27.932180  9163 raft_consensus.cc:515] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.933693  9163 leader_election.cc:290] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:27.934492  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ad52027db5443b9e53187cfcbbf9e3" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:27.934849  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ad52027db5443b9e53187cfcbbf9e3" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:27.935163  9090 raft_consensus.cc:3055] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:27.935443  9239 raft_consensus.cc:3060] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:27.935526  9090 raft_consensus.cc:740] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:27.936404  9090 consensus_queue.cc:260] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.937672  9090 raft_consensus.cc:3060] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:27.942934  9239 raft_consensus.cc:2468] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:27.944095  9128 leader_election.cc:304] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:27.944885  9286 raft_consensus.cc:2804] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:27.944917  9090 raft_consensus.cc:2468] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:27.945376  9286 raft_consensus.cc:697] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:27.946260  9286 consensus_queue.cc:237] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:27.952123  8988 catalog_manager.cc:5697] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 1 to 2, leader changed from 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65) to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 2 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.959663  9305 consensus_queue.cc:1048] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.970605  9288 consensus_queue.cc:1048] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.973732  9329 consensus_queue.cc:1048] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.988121  9351 consensus_queue.cc:1048] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:27.993034  9326 consensus_queue.cc:1048] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.002002  9289 raft_consensus.cc:993] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a: : Instructing follower 3f93ba6f3ac741ee93fa5765a94326e5 to start an election
I20260709 12:28:28.002501  9289 raft_consensus.cc:1081] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Signalling peer 3f93ba6f3ac741ee93fa5765a94326e5 to start an election
I20260709 12:28:28.004168  9090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2920b7e69b24a33bde09c66cd1a8db9"
dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
 from {username='slave'} at 127.0.0.1:34102
I20260709 12:28:28.004760  9090 raft_consensus.cc:493] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:28.005153  9090 raft_consensus.cc:3060] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.006633  9326 consensus_queue.cc:1048] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.013602  9090 raft_consensus.cc:515] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:28.016209  9090 leader_election.cc:290] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 2 election: Requested vote from peers 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:28.017340  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2920b7e69b24a33bde09c66cd1a8db9" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:28.017999  9165 raft_consensus.cc:3060] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.016839  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2920b7e69b24a33bde09c66cd1a8db9" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:28.018908  9305 raft_consensus.cc:993] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5: : Instructing follower 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:28.019381  9239 raft_consensus.cc:3055] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:28.019394  9326 raft_consensus.cc:1081] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Signalling peer 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:28.019972  9239 raft_consensus.cc:740] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:28.021080  9239 consensus_queue.cc:260] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:28.023325  9163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c01405fa79f45278781c477b429c667"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:58188
I20260709 12:28:28.024010  9163 raft_consensus.cc:493] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:28.024349  9163 raft_consensus.cc:3060] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.025108  9165 raft_consensus.cc:2468] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 2.
I20260709 12:28:28.023419  9239 raft_consensus.cc:3060] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.026736  9053 leader_election.cc:304] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:28.027527  9326 raft_consensus.cc:2804] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:28.027998  9326 raft_consensus.cc:697] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:28.028820  9326 consensus_queue.cc:237] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:28.030699  9163 raft_consensus.cc:515] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.032742  9163 leader_election.cc:290] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:28.033030  9239 raft_consensus.cc:2468] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 2.
I20260709 12:28:28.035359  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c01405fa79f45278781c477b429c667" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:28.035368  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c01405fa79f45278781c477b429c667" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:28.036118  9239 raft_consensus.cc:3060] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.036188  9090 raft_consensus.cc:3055] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:28.036664  9090 raft_consensus.cc:740] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:28.037237  8988 catalog_manager.cc:5697] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 1 to 2, leader changed from 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67) to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 2 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.037488  9090 consensus_queue.cc:260] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.038870  9090 raft_consensus.cc:3060] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.043044  9239 raft_consensus.cc:2468] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:28.044294  9128 leader_election.cc:304] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:28.045102  9286 raft_consensus.cc:2804] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:28.045521  9286 raft_consensus.cc:697] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:28.045866  9090 raft_consensus.cc:2468] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:28.046293  9286 consensus_queue.cc:237] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.052373  8988 catalog_manager.cc:5697] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 1 to 2, leader changed from 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65) to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 2 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.147679  9286 consensus_queue.cc:1048] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.158475  9299 consensus_queue.cc:1048] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:28.165611  9286 raft_consensus.cc:993] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f: : Instructing follower 3f93ba6f3ac741ee93fa5765a94326e5 to start an election
I20260709 12:28:28.166070  9299 raft_consensus.cc:1081] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Signalling peer 3f93ba6f3ac741ee93fa5765a94326e5 to start an election
I20260709 12:28:28.167616  9090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a77ba1e532f485abf98c54769535556"
dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
 from {username='slave'} at 127.0.0.1:34086
I20260709 12:28:28.168095  9090 raft_consensus.cc:493] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:28.168395  9090 raft_consensus.cc:3060] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.173568  9090 raft_consensus.cc:515] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.175472  9090 leader_election.cc:290] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 2 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:28.176160  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a77ba1e532f485abf98c54769535556" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:28.176596  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a77ba1e532f485abf98c54769535556" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:28.176826  9163 raft_consensus.cc:3055] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:28.177207  9163 raft_consensus.cc:740] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:28.177232  9239 raft_consensus.cc:3060] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.178169  9163 consensus_queue.cc:260] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.179471  9163 raft_consensus.cc:3060] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.184571  9239 raft_consensus.cc:2468] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 2.
I20260709 12:28:28.185839  9053 leader_election.cc:304] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:28.186584  9163 raft_consensus.cc:2468] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 2.
I20260709 12:28:28.186627  9326 raft_consensus.cc:2804] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:28.187211  9326 raft_consensus.cc:697] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:28.188056  9326 consensus_queue.cc:237] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.193683  8987 catalog_manager.cc:5697] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 1 to 2, leader changed from 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66) to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 2 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.330530  9090 raft_consensus.cc:1275] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.331857  9299 consensus_queue.cc:1048] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.339491  9239 raft_consensus.cc:1275] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.341676  9299 consensus_queue.cc:1048] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.373943  9337 raft_consensus.cc:993] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f: : Instructing follower 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:28.374550  9299 raft_consensus.cc:1081] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Signalling peer 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:28.376163  9239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04120eea993a4a4c948f77e924c1cd39"
dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:38990
I20260709 12:28:28.376688  9239 raft_consensus.cc:493] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:28.377058  9239 raft_consensus.cc:3060] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.382138  9239 raft_consensus.cc:515] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.384014  9239 leader_election.cc:290] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:28.384883  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04120eea993a4a4c948f77e924c1cd39" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:28.385147  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04120eea993a4a4c948f77e924c1cd39" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:28.385476  9090 raft_consensus.cc:3060] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.385816  9163 raft_consensus.cc:3055] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:28.386108  9163 raft_consensus.cc:740] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:28.386886  9163 consensus_queue.cc:260] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.387848  9163 raft_consensus.cc:3060] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:28.390763  9090 raft_consensus.cc:2468] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 2.
I20260709 12:28:28.391912  9202 leader_election.cc:304] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:28.392624  9351 raft_consensus.cc:2804] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:28.393138  9351 raft_consensus.cc:697] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:28.393365  9163 raft_consensus.cc:2468] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 2.
I20260709 12:28:28.393976  9351 consensus_queue.cc:237] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:28.400728  8988 catalog_manager.cc:5697] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 1 to 2, leader changed from 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66) to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 2 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.457105  9239 raft_consensus.cc:1275] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Refusing update from remote peer 3f93ba6f3ac741ee93fa5765a94326e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.458498  9326 consensus_queue.cc:1048] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.466154  9163 raft_consensus.cc:1275] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Refusing update from remote peer 3f93ba6f3ac741ee93fa5765a94326e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.467708  9326 consensus_queue.cc:1048] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.470038  9090 raft_consensus.cc:1275] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.471587  9299 consensus_queue.cc:1048] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.481566  9239 raft_consensus.cc:1275] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.483347  9337 consensus_queue.cc:1048] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.706578  9239 raft_consensus.cc:1275] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Refusing update from remote peer 3f93ba6f3ac741ee93fa5765a94326e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.707893  9288 consensus_queue.cc:1048] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.715924  9163 raft_consensus.cc:1275] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Refusing update from remote peer 3f93ba6f3ac741ee93fa5765a94326e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.718991  9305 consensus_queue.cc:1048] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.806176  9090 raft_consensus.cc:1275] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.807437  9351 consensus_queue.cc:1048] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:28.816733  9163 raft_consensus.cc:1275] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:28.818490  9351 consensus_queue.cc:1048] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:30.951133  8987 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37740:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:30.953240  8987 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:31.003360  9080 tablet_service.cc:1511] Processing CreateTablet for tablet 98bec205679d4737b1136c2050637373 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:31.004662  9080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98bec205679d4737b1136c2050637373. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.005021  9150 tablet_service.cc:1511] Processing CreateTablet for tablet 98bec205679d4737b1136c2050637373 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:31.006379  9150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98bec205679d4737b1136c2050637373. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.007184  9223 tablet_service.cc:1511] Processing CreateTablet for tablet 98bec205679d4737b1136c2050637373 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:31.008492  9223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98bec205679d4737b1136c2050637373. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.009649  9152 tablet_service.cc:1511] Processing CreateTablet for tablet 75aac328039c40c08f8a13b0b78c6307 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:31.010995  9152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75aac328039c40c08f8a13b0b78c6307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.011298  9222 tablet_service.cc:1511] Processing CreateTablet for tablet 75aac328039c40c08f8a13b0b78c6307 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:31.012636  9222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75aac328039c40c08f8a13b0b78c6307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.014564  9149 tablet_service.cc:1511] Processing CreateTablet for tablet d9a991ef86b041acb942f1842ccbe254 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:31.015836  9149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9a991ef86b041acb942f1842ccbe254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.019526  9078 tablet_service.cc:1511] Processing CreateTablet for tablet d9a991ef86b041acb942f1842ccbe254 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:31.020903  9075 tablet_service.cc:1511] Processing CreateTablet for tablet 75aac328039c40c08f8a13b0b78c6307 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:31.024379  9078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9a991ef86b041acb942f1842ccbe254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.026813  9075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75aac328039c40c08f8a13b0b78c6307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.036634  9226 tablet_service.cc:1511] Processing CreateTablet for tablet d9a991ef86b041acb942f1842ccbe254 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:31.039237  9155 tablet_service.cc:1511] Processing CreateTablet for tablet 57de5299d7574277a4f32823f094feb2 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:31.040571  9155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57de5299d7574277a4f32823f094feb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.041466  9225 tablet_service.cc:1511] Processing CreateTablet for tablet 57de5299d7574277a4f32823f094feb2 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:31.043004  9076 tablet_service.cc:1511] Processing CreateTablet for tablet 57de5299d7574277a4f32823f094feb2 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:31.043644  9226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9a991ef86b041acb942f1842ccbe254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.044270  9076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57de5299d7574277a4f32823f094feb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.045526  9371 tablet_bootstrap.cc:492] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.046872  9225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57de5299d7574277a4f32823f094feb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.053332  9371 tablet_bootstrap.cc:654] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.057942  9074 tablet_service.cc:1511] Processing CreateTablet for tablet 2da3589398f44c809c760886d4f1879e (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:31.059556  9074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2da3589398f44c809c760886d4f1879e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.063665  9154 tablet_service.cc:1511] Processing CreateTablet for tablet 2da3589398f44c809c760886d4f1879e (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:31.065442  9154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2da3589398f44c809c760886d4f1879e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.074123  9373 tablet_bootstrap.cc:492] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:31.082003  9373 tablet_bootstrap.cc:654] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.085871  9226 tablet_service.cc:1511] Processing CreateTablet for tablet 2da3589398f44c809c760886d4f1879e (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:31.087677  9226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2da3589398f44c809c760886d4f1879e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.097944  9080 tablet_service.cc:1511] Processing CreateTablet for tablet e7cb21b02d9b4b3bb1d156baca45ec48 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:31.099263  9080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7cb21b02d9b4b3bb1d156baca45ec48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.113494  9153 tablet_service.cc:1511] Processing CreateTablet for tablet e7cb21b02d9b4b3bb1d156baca45ec48 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:31.114918  9153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7cb21b02d9b4b3bb1d156baca45ec48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.118522  9224 tablet_service.cc:1511] Processing CreateTablet for tablet e7cb21b02d9b4b3bb1d156baca45ec48 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:31.120107  9224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7cb21b02d9b4b3bb1d156baca45ec48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.125315  9151 tablet_service.cc:1511] Processing CreateTablet for tablet 257bddd4b4564ccc9b17f16c85d7fc39 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:31.127049  9151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257bddd4b4564ccc9b17f16c85d7fc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.134379  9080 tablet_service.cc:1511] Processing CreateTablet for tablet 257bddd4b4564ccc9b17f16c85d7fc39 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:31.135705  9080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257bddd4b4564ccc9b17f16c85d7fc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.162204  9229 tablet_service.cc:1511] Processing CreateTablet for tablet 257bddd4b4564ccc9b17f16c85d7fc39 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:31.163668  9229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 257bddd4b4564ccc9b17f16c85d7fc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.167500  9077 tablet_service.cc:1511] Processing CreateTablet for tablet e08151a3778945128367bc763a6f13ad (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:31.172727  9077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08151a3778945128367bc763a6f13ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.177276  9148 tablet_service.cc:1511] Processing CreateTablet for tablet e08151a3778945128367bc763a6f13ad (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:31.178681  9148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08151a3778945128367bc763a6f13ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.184387  9228 tablet_service.cc:1511] Processing CreateTablet for tablet e08151a3778945128367bc763a6f13ad (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:31.185691  9228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08151a3778945128367bc763a6f13ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.191632  9147 tablet_service.cc:1511] Processing CreateTablet for tablet 47012003b93e4a039b6747b9d3826856 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:31.193387  9147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47012003b93e4a039b6747b9d3826856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.194073  9226 tablet_service.cc:1511] Processing CreateTablet for tablet 47012003b93e4a039b6747b9d3826856 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:31.191629  9072 tablet_service.cc:1511] Processing CreateTablet for tablet 47012003b93e4a039b6747b9d3826856 (DEFAULT_TABLE table=soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:31.195448  9226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47012003b93e4a039b6747b9d3826856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.196035  9072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47012003b93e4a039b6747b9d3826856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:31.228220  9376 tablet_bootstrap.cc:492] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
W20260709 12:28:31.257651  9036 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:31.319832  9376 tablet_bootstrap.cc:654] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.321537  9373 tablet_bootstrap.cc:492] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:31.319864  9371 tablet_bootstrap.cc:492] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.322078  9373 ts_tablet_manager.cc:1403] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.248s	user 0.032s	sys 0.120s
I20260709 12:28:31.322242  9371 ts_tablet_manager.cc:1403] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.277s	user 0.072s	sys 0.100s
I20260709 12:28:31.325027  9373 raft_consensus.cc:359] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.325465  9371 raft_consensus.cc:359] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.326313  9371 raft_consensus.cc:385] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.326617  9371 raft_consensus.cc:740] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.327322  9371 consensus_queue.cc:260] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.329480  9373 raft_consensus.cc:385] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.329835  9373 raft_consensus.cc:740] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.329713  9371 ts_tablet_manager.cc:1434] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:28:31.330782  9373 consensus_queue.cc:260] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.331337  9371 tablet_bootstrap.cc:492] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.333106  9373 ts_tablet_manager.cc:1434] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 12:28:31.338456  9373 tablet_bootstrap.cc:492] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:31.341280  9371 tablet_bootstrap.cc:654] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.344506  9376 tablet_bootstrap.cc:492] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.345135  9373 tablet_bootstrap.cc:654] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.345063  9376 ts_tablet_manager.cc:1403] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.117s	user 0.012s	sys 0.016s
I20260709 12:28:31.352759  9376 raft_consensus.cc:359] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.353621  9376 raft_consensus.cc:385] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.353969  9376 raft_consensus.cc:740] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.354759  9376 consensus_queue.cc:260] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.355228  9371 tablet_bootstrap.cc:492] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.355919  9371 ts_tablet_manager.cc:1403] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.014s
I20260709 12:28:31.358918  9371 raft_consensus.cc:359] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.359629  9371 raft_consensus.cc:385] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.359962  9371 raft_consensus.cc:740] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.360795  9371 consensus_queue.cc:260] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.363101  9371 ts_tablet_manager.cc:1434] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:31.364231  9371 tablet_bootstrap.cc:492] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.365443  9376 ts_tablet_manager.cc:1434] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:28:31.366005  9373 tablet_bootstrap.cc:492] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:31.366571  9373 ts_tablet_manager.cc:1403] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.015s
I20260709 12:28:31.368484  9376 tablet_bootstrap.cc:492] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:31.369668  9373 raft_consensus.cc:359] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.370496  9373 raft_consensus.cc:385] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.370810  9373 raft_consensus.cc:740] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.371780  9373 consensus_queue.cc:260] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.370915  9371 tablet_bootstrap.cc:654] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.375993  9376 tablet_bootstrap.cc:654] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.380326  9371 tablet_bootstrap.cc:492] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.380810  9371 ts_tablet_manager.cc:1403] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 12:28:31.382354  9373 ts_tablet_manager.cc:1434] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:28:31.383816  9371 raft_consensus.cc:359] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.384627  9371 raft_consensus.cc:385] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.384950  9371 raft_consensus.cc:740] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.385275  9373 tablet_bootstrap.cc:492] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:31.385757  9371 consensus_queue.cc:260] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.387724  9376 tablet_bootstrap.cc:492] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.389169  9326 raft_consensus.cc:493] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.389686  9326 raft_consensus.cc:515] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.389998  9371 ts_tablet_manager.cc:1434] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:28:31.392052  9326 leader_election.cc:290] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.392869  9371 tablet_bootstrap.cc:492] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.393086  9373 tablet_bootstrap.cc:654] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.388267  9376 ts_tablet_manager.cc:1403] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 12:28:31.396392  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:31.397507  9376 raft_consensus.cc:359] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.401808  9371 tablet_bootstrap.cc:654] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.402115  9376 raft_consensus.cc:385] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.403034  9376 raft_consensus.cc:740] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.399118  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:31.404254  9376 consensus_queue.cc:260] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
W20260709 12:28:31.398007  9053 leader_election.cc:343] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:31.405853  9053 leader_election.cc:343] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.406322  9053 leader_election.cc:304] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:31.407841  9326 raft_consensus.cc:2749] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:31.413157  9376 ts_tablet_manager.cc:1434] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:28:31.414283  9376 tablet_bootstrap.cc:492] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:31.421968  9373 tablet_bootstrap.cc:492] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:31.422640  9373 ts_tablet_manager.cc:1403] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.038s	user 0.024s	sys 0.011s
I20260709 12:28:31.422852  9376 tablet_bootstrap.cc:654] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.424371  9371 tablet_bootstrap.cc:492] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.424932  9371 ts_tablet_manager.cc:1403] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.012s
I20260709 12:28:31.425866  9373 raft_consensus.cc:359] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.426729  9373 raft_consensus.cc:385] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.427073  9373 raft_consensus.cc:740] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.427932  9371 raft_consensus.cc:359] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.428040  9373 consensus_queue.cc:260] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.428797  9371 raft_consensus.cc:385] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.429168  9371 raft_consensus.cc:740] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.429967  9371 consensus_queue.cc:260] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.430824  9373 ts_tablet_manager.cc:1434] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:28:31.432322  9371 ts_tablet_manager.cc:1434] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:31.433406  9371 tablet_bootstrap.cc:492] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.436076  9373 tablet_bootstrap.cc:492] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:31.440404  9371 tablet_bootstrap.cc:654] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.445053  9373 tablet_bootstrap.cc:654] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.451195  9376 tablet_bootstrap.cc:492] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.451910  9376 ts_tablet_manager.cc:1403] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.013s
I20260709 12:28:31.453090  9373 tablet_bootstrap.cc:492] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:31.453687  9373 ts_tablet_manager.cc:1403] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.000s
I20260709 12:28:31.456126  9376 raft_consensus.cc:359] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.457276  9376 raft_consensus.cc:385] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.457774  9376 raft_consensus.cc:740] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.458828  9376 consensus_queue.cc:260] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.456902  9373 raft_consensus.cc:359] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.459997  9373 raft_consensus.cc:385] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.460362  9373 raft_consensus.cc:740] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.461558  9376 ts_tablet_manager.cc:1434] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:28:31.462666  9371 tablet_bootstrap.cc:492] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.463567  9371 ts_tablet_manager.cc:1403] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.009s
I20260709 12:28:31.461194  9373 consensus_queue.cc:260] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.466732  9371 raft_consensus.cc:359] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.467494  9371 raft_consensus.cc:385] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.467821  9371 raft_consensus.cc:740] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.468046  9373 ts_tablet_manager.cc:1434] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.005s
I20260709 12:28:31.469170  9373 tablet_bootstrap.cc:492] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:31.468643  9371 consensus_queue.cc:260] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.462733  9376 tablet_bootstrap.cc:492] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:31.473937  9371 ts_tablet_manager.cc:1434] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:28:31.475035  9375 raft_consensus.cc:493] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.475553  9375 raft_consensus.cc:515] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.476485  9373 tablet_bootstrap.cc:654] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.477183  9376 tablet_bootstrap.cc:654] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.477824  9375 leader_election.cc:290] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.475066  9371 tablet_bootstrap.cc:492] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.479803  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aac328039c40c08f8a13b0b78c6307" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
W20260709 12:28:31.481678  9053 leader_election.cc:343] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.482197  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aac328039c40c08f8a13b0b78c6307" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:31.482967  9239 raft_consensus.cc:2468] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:31.484802  9053 leader_election.cc:304] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:31.490038  9376 tablet_bootstrap.cc:492] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.490679  9375 raft_consensus.cc:2804] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:31.490933  9376 ts_tablet_manager.cc:1403] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.010s
I20260709 12:28:31.491148  9375 raft_consensus.cc:493] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.491750  9375 raft_consensus.cc:3060] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.495537  9376 raft_consensus.cc:359] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.496701  9376 raft_consensus.cc:385] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.497133  9376 raft_consensus.cc:740] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.498878  9376 consensus_queue.cc:260] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.499914  9375 raft_consensus.cc:515] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.502204  9375 leader_election.cc:290] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.490370  9371 tablet_bootstrap.cc:654] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.503669  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aac328039c40c08f8a13b0b78c6307" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:31.507064  9376 ts_tablet_manager.cc:1434] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 12:28:31.507603  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75aac328039c40c08f8a13b0b78c6307" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:31.508235  9239 raft_consensus.cc:3060] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.508716  9375 raft_consensus.cc:493] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.509462  9376 tablet_bootstrap.cc:492] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:31.509269  9375 raft_consensus.cc:515] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
W20260709 12:28:31.511058  9053 leader_election.cc:343] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.513576  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:31.515286  9373 tablet_bootstrap.cc:492] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:31.515861  9373 ts_tablet_manager.cc:1403] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.024s
I20260709 12:28:31.517110  9376 tablet_bootstrap.cc:654] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.518304  9238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
W20260709 12:28:31.518579  9053 leader_election.cc:343] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:31.519555  9053 leader_election.cc:343] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.519402  9373 raft_consensus.cc:359] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.522475  9373 raft_consensus.cc:385] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.522475  9053 leader_election.cc:304] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:31.523084  9373 raft_consensus.cc:740] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.523361  9371 tablet_bootstrap.cc:492] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.523988  9326 raft_consensus.cc:2749] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:31.524077  9371 ts_tablet_manager.cc:1403] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.001s
I20260709 12:28:31.524124  9373 consensus_queue.cc:260] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.526952  9239 raft_consensus.cc:2468] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:31.527427  9371 raft_consensus.cc:359] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.528190  9371 raft_consensus.cc:385] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.511446  9375 leader_election.cc:290] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.528414  9053 leader_election.cc:304] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:31.528482  9371 raft_consensus.cc:740] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.526991  9373 ts_tablet_manager.cc:1434] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:28:31.530006  9371 consensus_queue.cc:260] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.531013  9373 tablet_bootstrap.cc:492] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:31.531204  9326 raft_consensus.cc:2804] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:31.532550  9371 ts_tablet_manager.cc:1434] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:28:31.533567  9371 tablet_bootstrap.cc:492] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.536579  9376 tablet_bootstrap.cc:492] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.537173  9376 ts_tablet_manager.cc:1403] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.005s
I20260709 12:28:31.540102  9371 tablet_bootstrap.cc:654] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.540252  9376 raft_consensus.cc:359] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.541002  9376 raft_consensus.cc:385] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.541296  9376 raft_consensus.cc:740] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.542166  9376 consensus_queue.cc:260] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.543107  9373 tablet_bootstrap.cc:654] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.548599  9376 ts_tablet_manager.cc:1434] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:28:31.550020  9376 tablet_bootstrap.cc:492] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:31.558815  9376 tablet_bootstrap.cc:654] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.572858  9376 tablet_bootstrap.cc:492] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.573419  9376 ts_tablet_manager.cc:1403] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.011s
I20260709 12:28:31.576768  9376 raft_consensus.cc:359] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.577567  9376 raft_consensus.cc:385] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.577919  9376 raft_consensus.cc:740] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.578847  9376 consensus_queue.cc:260] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.585321  9376 ts_tablet_manager.cc:1434] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:28:31.586486  9376 tablet_bootstrap.cc:492] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:31.593888  9376 tablet_bootstrap.cc:654] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.605350  9326 raft_consensus.cc:697] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:31.606551  9326 consensus_queue.cc:237] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.603852  9375 raft_consensus.cc:493] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.613339  9375 raft_consensus.cc:515] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.621600  9375 leader_election.cc:290] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.623142  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9a991ef86b041acb942f1842ccbe254" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:31.624429  9165 raft_consensus.cc:2468] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:31.624840  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9a991ef86b041acb942f1842ccbe254" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:31.625700  9239 raft_consensus.cc:2468] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:31.625856  9053 leader_election.cc:304] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:31.628144  9376 tablet_bootstrap.cc:492] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.629002  9375 raft_consensus.cc:2804] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:31.629874  9376 ts_tablet_manager.cc:1403] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.008s
I20260709 12:28:31.630890  9286 raft_consensus.cc:493] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.631614  9286 raft_consensus.cc:515] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.633510  9376 raft_consensus.cc:359] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.634382  9376 raft_consensus.cc:385] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.634900  9376 raft_consensus.cc:740] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.635720  9376 consensus_queue.cc:260] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.638005  9375 raft_consensus.cc:493] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.638612  9375 raft_consensus.cc:3060] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.640355  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57de5299d7574277a4f32823f094feb2" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
W20260709 12:28:31.642170  9128 leader_election.cc:343] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.642448  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57de5299d7574277a4f32823f094feb2" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:31.634295  9286 leader_election.cc:290] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.646909  9375 raft_consensus.cc:515] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
W20260709 12:28:31.649194  9128 leader_election.cc:343] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.649930  9128 leader_election.cc:304] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5
I20260709 12:28:31.650027  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9a991ef86b041acb942f1842ccbe254" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:31.650863  9165 raft_consensus.cc:3060] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.651043  9376 ts_tablet_manager.cc:1434] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:28:31.651701  9375 leader_election.cc:290] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.653643  9373 tablet_bootstrap.cc:492] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:31.654273  9373 ts_tablet_manager.cc:1403] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.124s	user 0.038s	sys 0.041s
I20260709 12:28:31.654516  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9a991ef86b041acb942f1842ccbe254" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:31.655099  9239 raft_consensus.cc:3060] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.655654  9376 tablet_bootstrap.cc:492] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:31.655174  8986 catalog_manager.cc:5697] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:31.656754  9371 tablet_bootstrap.cc:492] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.661190  9371 ts_tablet_manager.cc:1403] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.128s	user 0.074s	sys 0.048s
I20260709 12:28:31.657711  9373 raft_consensus.cc:359] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.662892  9165 raft_consensus.cc:2468] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:31.663028  9373 raft_consensus.cc:385] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.663537  9373 raft_consensus.cc:740] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.664294  9053 leader_election.cc:304] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:31.664237  9371 raft_consensus.cc:359] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.665167  9371 raft_consensus.cc:385] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.665326  9375 raft_consensus.cc:2804] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:31.665516  9371 raft_consensus.cc:740] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.665899  9375 raft_consensus.cc:697] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:31.664552  9373 consensus_queue.cc:260] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.666347  9371 consensus_queue.cc:260] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.669888  9376 tablet_bootstrap.cc:654] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.666990  9375 consensus_queue.cc:237] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.671797  9371 ts_tablet_manager.cc:1434] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:28:31.672880  9371 tablet_bootstrap.cc:492] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.654559  9286 raft_consensus.cc:2749] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:31.681499  9239 raft_consensus.cc:2468] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:31.688602  9371 tablet_bootstrap.cc:654] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.690135  9373 ts_tablet_manager.cc:1434] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.003s
I20260709 12:28:31.698844  9373 tablet_bootstrap.cc:492] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:31.698896  9376 tablet_bootstrap.cc:492] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.699293  9371 tablet_bootstrap.cc:492] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.699731  9376 ts_tablet_manager.cc:1403] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.045s	user 0.027s	sys 0.012s
I20260709 12:28:31.699796  9371 ts_tablet_manager.cc:1403] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.002s
I20260709 12:28:31.702883  9376 raft_consensus.cc:359] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.703799  9376 raft_consensus.cc:385] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.704141  9376 raft_consensus.cc:740] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.704974  9376 consensus_queue.cc:260] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.705739  9373 tablet_bootstrap.cc:654] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.706156  9329 raft_consensus.cc:493] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.706677  9329 raft_consensus.cc:515] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.707681  9376 ts_tablet_manager.cc:1434] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:31.708938  9376 tablet_bootstrap.cc:492] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f: Bootstrap starting.
I20260709 12:28:31.708839  9329 leader_election.cc:290] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:31.711418  9286 raft_consensus.cc:493] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.711418  9371 raft_consensus.cc:359] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.712406  9371 raft_consensus.cc:385] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.712258  9286 raft_consensus.cc:515] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.712687  9371 raft_consensus.cc:740] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.712815  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:31.713534  9371 consensus_queue.cc:260] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.713743  9165 raft_consensus.cc:2468] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 0.
I20260709 12:28:31.715118  9286 leader_election.cc:290] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.715416  9371 ts_tablet_manager.cc:1434] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:28:31.716466  9371 tablet_bootstrap.cc:492] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5: Bootstrap starting.
I20260709 12:28:31.718389  9286 raft_consensus.cc:493] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.718876  9286 raft_consensus.cc:515] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.719193  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:31.720043  9090 raft_consensus.cc:2468] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:31.721274  9329 raft_consensus.cc:493] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.722735  9286 leader_election.cc:290] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.721758  9329 raft_consensus.cc:515] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.724144  9202 leader_election.cc:304] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:31.724858  9329 leader_election.cc:290] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:31.725651  9329 raft_consensus.cc:2804] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:31.726061  9376 tablet_bootstrap.cc:654] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.726207  9329 raft_consensus.cc:493] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.726609  9329 raft_consensus.cc:3060] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.711525  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:31.722505  9128 leader_election.cc:304] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:31.731237  9371 tablet_bootstrap.cc:654] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:31.731534  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da3589398f44c809c760886d4f1879e" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:31.732645  9090 raft_consensus.cc:2468] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:31.737624  9128 leader_election.cc:304] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:31.740044  9382 raft_consensus.cc:2804] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:31.740476  9382 raft_consensus.cc:493] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.740873  9382 raft_consensus.cc:3060] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.754674  9375 raft_consensus.cc:493] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.755306  9375 raft_consensus.cc:515] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.755877  9089 raft_consensus.cc:2468] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 0.
I20260709 12:28:31.757694  9286 raft_consensus.cc:2804] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:31.758337  9286 raft_consensus.cc:493] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.758881  9286 raft_consensus.cc:3060] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.759557  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
W20260709 12:28:31.761395  9053 leader_election.cc:343] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.762166  9375 leader_election.cc:290] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.754120  9329 raft_consensus.cc:515] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.770498  9329 leader_election.cc:290] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:31.776608  9382 raft_consensus.cc:515] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
W20260709 12:28:31.778414  9202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.29.37.66:44727, user_credentials={real_user=slave}} blocked reactor thread for 54399.8us
I20260709 12:28:31.780197  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:31.782114  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da3589398f44c809c760886d4f1879e" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:31.782886  9239 raft_consensus.cc:2468] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 0.
I20260709 12:28:31.785085  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47012003b93e4a039b6747b9d3826856" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:31.788347  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47012003b93e4a039b6747b9d3826856" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:31.789147  9165 raft_consensus.cc:2468] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 0.
I20260709 12:28:31.791734  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:31.792548  9089 raft_consensus.cc:3060] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.799574  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:31.800437  9165 raft_consensus.cc:3060] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:31.813699  9128 leader_election.cc:343] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.823643  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:31.824548  9163 raft_consensus.cc:2468] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
W20260709 12:28:31.825320  9202 leader_election.cc:343] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.829340  9053 leader_election.cc:304] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:31.847898  9375 raft_consensus.cc:2804] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:31.848289  9375 raft_consensus.cc:493] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.848671  9375 raft_consensus.cc:3060] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.849272  9202 leader_election.cc:304] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 3f93ba6f3ac741ee93fa5765a94326e5
I20260709 12:28:31.850116  9405 raft_consensus.cc:2804] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:31.850597  9405 raft_consensus.cc:493] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.850958  9405 raft_consensus.cc:3060] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.852593  9165 raft_consensus.cc:2468] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 1.
I20260709 12:28:31.844225  9382 leader_election.cc:290] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.861316  9089 raft_consensus.cc:2468] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 1.
I20260709 12:28:31.871483  8986 catalog_manager.cc:5697] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:31.876818  9375 raft_consensus.cc:515] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.841517  9286 raft_consensus.cc:515] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.891162  9202 leader_election.cc:304] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:31.891399  9376 tablet_bootstrap.cc:492] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f: No bootstrap required, opened a new log
I20260709 12:28:31.892027  9329 raft_consensus.cc:2804] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:31.892295  9376 ts_tablet_manager.cc:1403] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent bootstrapping tablet: real 0.184s	user 0.044s	sys 0.040s
W20260709 12:28:31.886902  9128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:38619, user_credentials={real_user=slave}} blocked reactor thread for 56691.3us
I20260709 12:28:31.896503  9286 leader_election.cc:290] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.897892  9405 raft_consensus.cc:515] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.901365  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:31.901518  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47012003b93e4a039b6747b9d3826856" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:31.901954  9090 raft_consensus.cc:3060] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.899892  9405 leader_election.cc:290] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:31.906479  9375 leader_election.cc:290] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.907517  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:31.908219  9165 raft_consensus.cc:3060] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.908519  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da3589398f44c809c760886d4f1879e" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:31.909379  9089 raft_consensus.cc:3060] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.910544  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
W20260709 12:28:31.912209  9053 leader_election.cc:343] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:31.915531  9202 leader_election.cc:343] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.917184  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:31.930709  9090 raft_consensus.cc:2468] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:31.931907  9165 raft_consensus.cc:2468] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
W20260709 12:28:31.935657  9128 leader_election.cc:343] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.936874  9053 leader_election.cc:304] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:31.932397  9405 raft_consensus.cc:493] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.932612  9371 tablet_bootstrap.cc:492] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5: No bootstrap required, opened a new log
I20260709 12:28:31.934374  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47012003b93e4a039b6747b9d3826856" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:31.938319  9405 raft_consensus.cc:515] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.938849  9163 raft_consensus.cc:3060] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.941041  9405 leader_election.cc:290] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:31.946630  9371 ts_tablet_manager.cc:1403] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent bootstrapping tablet: real 0.230s	user 0.027s	sys 0.055s
I20260709 12:28:31.937803  9375 raft_consensus.cc:2804] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:31.951577  9375 raft_consensus.cc:697] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:31.952757  9375 consensus_queue.cc:237] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.959262  9329 raft_consensus.cc:697] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:31.960631  9329 consensus_queue.cc:237] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.962327  9128 leader_election.cc:304] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:31.963974  9410 raft_consensus.cc:2804] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:31.964461  9410 raft_consensus.cc:697] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:31.964897  9089 raft_consensus.cc:2468] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:31.965320  9410 consensus_queue.cc:237] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.968065  9371 raft_consensus.cc:359] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.968780  9371 raft_consensus.cc:385] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.969064  9371 raft_consensus.cc:740] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.969767  9371 consensus_queue.cc:260] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.970964  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2da3589398f44c809c760886d4f1879e" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:31.971093  9165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08151a3778945128367bc763a6f13ad" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:31.971599  9239 raft_consensus.cc:3060] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.971717  9165 raft_consensus.cc:2468] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 0.
I20260709 12:28:31.971954  9371 ts_tablet_manager.cc:1434] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:28:31.972386  9373 tablet_bootstrap.cc:492] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:31.957863  9163 raft_consensus.cc:2468] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 1.
I20260709 12:28:31.973016  9373 ts_tablet_manager.cc:1403] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.274s	user 0.091s	sys 0.142s
I20260709 12:28:31.973445  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08151a3778945128367bc763a6f13ad" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:31.974148  9089 raft_consensus.cc:2468] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 0.
I20260709 12:28:31.977141  9128 leader_election.cc:304] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:31.960706  9376 raft_consensus.cc:359] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.979221  9376 raft_consensus.cc:385] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.978067  9411 raft_consensus.cc:2804] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:31.979632  9376 raft_consensus.cc:740] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Initialized, Role: FOLLOWER
I20260709 12:28:31.981910  9373 raft_consensus.cc:359] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.982184  9239 raft_consensus.cc:2468] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 1.
I20260709 12:28:31.985383  9411 raft_consensus.cc:697] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:31.986357  9411 consensus_queue.cc:237] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.985111  9202 leader_election.cc:304] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:31.988751  9405 raft_consensus.cc:2804] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:31.989107  9405 raft_consensus.cc:493] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.989462  9405 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:31.991194  9326 raft_consensus.cc:493] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.990922  9376 consensus_queue.cc:260] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.991711  9326 raft_consensus.cc:515] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:31.993736  9326 leader_election.cc:290] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:31.993902  9202 leader_election.cc:304] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 3f93ba6f3ac741ee93fa5765a94326e5
I20260709 12:28:31.995880  9412 raft_consensus.cc:2804] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:31.982963  9373 raft_consensus.cc:385] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:31.996977  8986 catalog_manager.cc:5697] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 1 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:31.999830  9164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57de5299d7574277a4f32823f094feb2" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:32.000656  9164 raft_consensus.cc:2468] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 0.
I20260709 12:28:32.001226  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57de5299d7574277a4f32823f094feb2" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:32.001929  9053 leader_election.cc:304] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
W20260709 12:28:32.002929  9053 leader_election.cc:343] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:31.997035  9373 raft_consensus.cc:740] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.003882  9326 raft_consensus.cc:2804] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:32.003649  8987 catalog_manager.cc:5697] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:32.004137  9373 consensus_queue.cc:260] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.004350  9326 raft_consensus.cc:493] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:32.006398  9326 raft_consensus.cc:3060] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.006052  9376 ts_tablet_manager.cc:1434] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.001s
I20260709 12:28:32.007457  9405 raft_consensus.cc:515] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.013830  9373 ts_tablet_manager.cc:1434] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:28:32.016333  9373 tablet_bootstrap.cc:492] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:32.017014  9412 raft_consensus.cc:697] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:32.017935  9412 consensus_queue.cc:237] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.018445  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08151a3778945128367bc763a6f13ad" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:32.019074  9089 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.017038  9326 raft_consensus.cc:515] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.023738  9164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57de5299d7574277a4f32823f094feb2" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:32.024353  9164 raft_consensus.cc:3060] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.024574  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57de5299d7574277a4f32823f094feb2" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
W20260709 12:28:32.025830  9053 leader_election.cc:343] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:32.026583  8985 catalog_manager.cc:5697] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 1 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:32.021628  8988 catalog_manager.cc:5697] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 1 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:32.030149  9089 raft_consensus.cc:2468] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 1.
I20260709 12:28:32.031545  9202 leader_election.cc:304] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:32.026579  9326 leader_election.cc:290] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:32.031874  9164 raft_consensus.cc:2468] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 1.
I20260709 12:28:32.033128  9053 leader_election.cc:304] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:32.033813  9418 raft_consensus.cc:2804] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:32.034263  9326 raft_consensus.cc:2804] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:32.034611  9418 raft_consensus.cc:697] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:32.034969  9326 raft_consensus.cc:697] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:32.035566  9418 consensus_queue.cc:237] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.035866  9326 consensus_queue.cc:237] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.036741  9373 tablet_bootstrap.cc:654] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:32.039479  9405 leader_election.cc:290] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:32.053726  9164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08151a3778945128367bc763a6f13ad" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:32.054457  9164 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.054733  9373 tablet_bootstrap.cc:492] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:32.055277  9373 ts_tablet_manager.cc:1403] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.014s
I20260709 12:28:32.057900  9373 raft_consensus.cc:359] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.058923  9373 raft_consensus.cc:385] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:32.059083  8986 catalog_manager.cc:5697] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:32.059198  9373 raft_consensus.cc:740] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.060854  9373 consensus_queue.cc:260] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.063401  9164 raft_consensus.cc:2468] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 1.
I20260709 12:28:32.064662  9373 ts_tablet_manager.cc:1434] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:32.065516  9373 tablet_bootstrap.cc:492] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a: Bootstrap starting.
I20260709 12:28:32.071264  9373 tablet_bootstrap.cc:654] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:32.072402  8988 catalog_manager.cc:5697] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 1 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:32.074170  8988 catalog_manager.cc:5697] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 1 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:32.076159  9373 tablet_bootstrap.cc:492] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a: No bootstrap required, opened a new log
I20260709 12:28:32.076648  9373 ts_tablet_manager.cc:1403] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent bootstrapping tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:28:32.078815  9373 raft_consensus.cc:359] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.079298  9373 raft_consensus.cc:385] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:32.079511  9373 raft_consensus.cc:740] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Initialized, Role: FOLLOWER
I20260709 12:28:32.080116  9373 consensus_queue.cc:260] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.081578  9373 ts_tablet_manager.cc:1434] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:28:32.084769  9412 raft_consensus.cc:493] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.085219  9412 raft_consensus.cc:515] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.087234  9412 leader_election.cc:290] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:32.087806  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57de5299d7574277a4f32823f094feb2" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:32.088233  9164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57de5299d7574277a4f32823f094feb2" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:32.089010  9164 raft_consensus.cc:2393] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 196d3384261d4a8dbf0a91f553b1ef4a in current term 1: Already voted for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in this term.
I20260709 12:28:32.089951  9202 leader_election.cc:304] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:32.090719  9412 raft_consensus.cc:3060] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.096517  9412 raft_consensus.cc:2749] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:32.129944  9326 consensus_queue.cc:1048] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.143792  9164 raft_consensus.cc:3060] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.152585  9426 consensus_queue.cc:1048] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:32.168080  9412 raft_consensus.cc:493] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.168613  9412 raft_consensus.cc:515] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.170607  9412 leader_election.cc:290] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:32.171157  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:32.171494  9164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:32.172369  9164 raft_consensus.cc:2393] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 196d3384261d4a8dbf0a91f553b1ef4a in current term 1: Already voted for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in this term.
I20260709 12:28:32.173298  9202 leader_election.cc:304] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:32.174057  9412 raft_consensus.cc:3060] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.178978  9412 raft_consensus.cc:2749] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:32.181442  9426 raft_consensus.cc:493] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.181905  9426 raft_consensus.cc:515] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.183678  9426 leader_election.cc:290] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:32.184439  9164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47012003b93e4a039b6747b9d3826856" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:32.184656  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47012003b93e4a039b6747b9d3826856" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" is_pre_election: true
I20260709 12:28:32.185266  9164 raft_consensus.cc:2393] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f93ba6f3ac741ee93fa5765a94326e5 in current term 1: Already voted for candidate 196d3384261d4a8dbf0a91f553b1ef4a in this term.
I20260709 12:28:32.186412  9053 leader_election.cc:304] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:32.187160  9426 raft_consensus.cc:3060] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.192960  9426 raft_consensus.cc:2749] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:32.227380  9426 consensus_queue.cc:1048] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.238684  9375 consensus_queue.cc:1048] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.289413  9163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98bec205679d4737b1136c2050637373"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
mode: GRACEFUL
new_leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
 from {username='slave'} at 127.0.0.1:58214
I20260709 12:28:32.289983  9163 raft_consensus.cc:606] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Received request to transfer leadership to TS 3f93ba6f3ac741ee93fa5765a94326e5
I20260709 12:28:32.303893  9089 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48"
dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
mode: GRACEFUL
new_leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:34110
I20260709 12:28:32.304352  9089 raft_consensus.cc:606] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Received request to transfer leadership to TS 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:32.308919  9239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39"
dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
mode: GRACEFUL
new_leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:39010
I20260709 12:28:32.309324  9239 raft_consensus.cc:606] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Received request to transfer leadership to TS 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:32.313869  9239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e08151a3778945128367bc763a6f13ad"
dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
mode: GRACEFUL
new_leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:39010
I20260709 12:28:32.314272  9239 raft_consensus.cc:606] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Received request to transfer leadership to TS 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:32.363240  9410 consensus_queue.cc:1048] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.368105  9412 raft_consensus.cc:493] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.368752  9412 raft_consensus.cc:515] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.370389  9375 consensus_queue.cc:1048] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.370952  9412 leader_election.cc:290] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:32.371522  9286 raft_consensus.cc:993] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: : Instructing follower 3f93ba6f3ac741ee93fa5765a94326e5 to start an election
I20260709 12:28:32.371965  9286 raft_consensus.cc:1081] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Signalling peer 3f93ba6f3ac741ee93fa5765a94326e5 to start an election
I20260709 12:28:32.374111  9090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98bec205679d4737b1136c2050637373"
dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
 from {username='slave'} at 127.0.0.1:34086
I20260709 12:28:32.374174  9088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" is_pre_election: true
I20260709 12:28:32.375820  9090 raft_consensus.cc:493] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:32.376210  9090 raft_consensus.cc:3060] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.378990  9238 raft_consensus.cc:3060] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:32.376224  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" is_pre_election: true
I20260709 12:28:32.380910  9202 leader_election.cc:304] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:32.384859  9090 raft_consensus.cc:515] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.387040  9412 raft_consensus.cc:2749] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:32.389230  9090 leader_election.cc:290] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 2 election: Requested vote from peers 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:32.390949  9164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:32.391605  9164 raft_consensus.cc:3055] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:32.391920  9164 raft_consensus.cc:740] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:32.392150  9237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98bec205679d4737b1136c2050637373" candidate_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:32.392803  9237 raft_consensus.cc:3060] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.392804  9164 consensus_queue.cc:260] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.396929  9375 consensus_queue.cc:1048] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:32.397841  9164 raft_consensus.cc:3060] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:32.398517  9410 consensus_queue.cc:1175] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" responder_term: 1 status { last_received { term: 0 index: 0 } last_committed_idx: 0 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:32.404567  9237 raft_consensus.cc:2468] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 2.
I20260709 12:28:32.408766  9382 consensus_queue.cc:1048] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.409485  9053 leader_election.cc:304] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:32.410692  9326 raft_consensus.cc:2804] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:32.411173  9326 raft_consensus.cc:697] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 LEADER]: Becoming Leader. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:32.411998  9326 consensus_queue.cc:237] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.416213  9326 raft_consensus.cc:993] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5: : Instructing follower 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:32.416693  9426 raft_consensus.cc:1081] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Signalling peer 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:32.420192  9163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:58188
I20260709 12:28:32.420884  9163 raft_consensus.cc:493] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:32.421274  9163 raft_consensus.cc:3060] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.421139  9418 consensus_queue.cc:1048] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.426168  9382 consensus_queue.cc:1048] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.426791  9164 raft_consensus.cc:2468] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f93ba6f3ac741ee93fa5765a94326e5 in term 2.
I20260709 12:28:32.429694  9163 raft_consensus.cc:515] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.430732  8988 catalog_manager.cc:5697] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 reported cstate change: term changed from 1 to 2, leader changed from 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66) to 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65). New cstate: current_term: 2 leader_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:32.431936  9163 leader_election.cc:290] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:32.432780  9088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:32.433358  9088 raft_consensus.cc:3055] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:32.433696  9088 raft_consensus.cc:740] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:32.434756  9088 consensus_queue.cc:260] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.436146  9088 raft_consensus.cc:3060] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.439881  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7cb21b02d9b4b3bb1d156baca45ec48" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:32.440518  9239 raft_consensus.cc:3060] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.447306  9239 raft_consensus.cc:2468] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:32.449546  9375 consensus_queue.cc:1048] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.453850  9128 leader_election.cc:304] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:32.455420  9286 raft_consensus.cc:2804] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:32.455845  9286 raft_consensus.cc:697] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:32.456562  9286 consensus_queue.cc:237] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.458923  9088 raft_consensus.cc:2468] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:32.465613  9375 consensus_queue.cc:1048] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.467881  8986 catalog_manager.cc:5697] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 1 to 2, leader changed from 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65) to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 2 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:32.478179  9412 consensus_queue.cc:1048] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.491509  9329 raft_consensus.cc:993] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a: : Instructing follower 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:32.492043  9413 raft_consensus.cc:1081] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Signalling peer 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:32.493506  9163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e08151a3778945128367bc763a6f13ad"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:58204
I20260709 12:28:32.494138  9163 raft_consensus.cc:493] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:32.494616  9163 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.501214  9163 raft_consensus.cc:515] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.503816  9090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08151a3778945128367bc763a6f13ad" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:32.503952  9238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08151a3778945128367bc763a6f13ad" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:32.504160  9163 leader_election.cc:290] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:32.504398  9090 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.504519  9238 raft_consensus.cc:3055] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:32.504925  9238 raft_consensus.cc:740] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:32.505828  9238 consensus_queue.cc:260] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.507092  9238 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.510461  9413 consensus_queue.cc:1048] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.511054  9090 raft_consensus.cc:2468] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:32.512231  9128 leader_election.cc:304] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:32.513123  9382 raft_consensus.cc:2804] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:32.513583  9382 raft_consensus.cc:697] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:32.514657  9382 consensus_queue.cc:237] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.517470  9238 raft_consensus.cc:2468] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:32.521816  9329 raft_consensus.cc:993] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a: : Instructing follower 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:32.522279  9412 raft_consensus.cc:1081] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Signalling peer 97b41bf0005c4e9c8a9ed4795fc9080f to start an election
I20260709 12:28:32.522089  9405 consensus_queue.cc:1048] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.524489  9163 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
 from {username='slave'} at 127.0.0.1:58204
I20260709 12:28:32.526116  9163 raft_consensus.cc:493] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:32.526539  9163 raft_consensus.cc:3060] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.533900  9163 raft_consensus.cc:515] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.536810  9163 leader_election.cc:290] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67:46713)
I20260709 12:28:32.537918  9088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:32.538161  9239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "257bddd4b4564ccc9b17f16c85d7fc39" candidate_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
I20260709 12:28:32.538790  9239 raft_consensus.cc:3055] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:32.539089  9239 raft_consensus.cc:740] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:32.539836  9239 consensus_queue.cc:260] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.540833  9239 raft_consensus.cc:3060] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.543752  8987 catalog_manager.cc:5697] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 1 to 2, leader changed from 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67) to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 2 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:32.549769  9239 raft_consensus.cc:2468] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b41bf0005c4e9c8a9ed4795fc9080f in term 2.
I20260709 12:28:32.550925  9128 leader_election.cc:304] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 3f93ba6f3ac741ee93fa5765a94326e5
I20260709 12:28:32.551700  9382 raft_consensus.cc:2804] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:32.552098  9382 raft_consensus.cc:697] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Becoming Leader. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:32.552700  9382 consensus_queue.cc:237] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:32.560261  8988 catalog_manager.cc:5697] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f reported cstate change: term changed from 1 to 2, leader changed from 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67) to 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66). New cstate: current_term: 2 leader_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:32.581878  9329 consensus_queue.cc:1048] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.593020  9405 consensus_queue.cc:1048] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.848222  9090 raft_consensus.cc:1275] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:32.849357  9382 consensus_queue.cc:1048] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:32.856719  9239 raft_consensus.cc:1275] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:32.858028  9286 consensus_queue.cc:1048] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:32.942448  9090 raft_consensus.cc:3060] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:32.947575  9090 raft_consensus.cc:1275] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:32.947956  9163 raft_consensus.cc:1275] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Refusing update from remote peer 3f93ba6f3ac741ee93fa5765a94326e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:32.948726  9286 consensus_queue.cc:1048] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:32.949391  9326 consensus_queue.cc:1048] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:32.958357  9239 raft_consensus.cc:1275] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:32.959951  9286 consensus_queue.cc:1048] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:32.961256  9239 raft_consensus.cc:1275] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Refusing update from remote peer 3f93ba6f3ac741ee93fa5765a94326e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:32.962903  9426 consensus_queue.cc:1048] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:32.979156  9237 raft_consensus.cc:1275] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:32.980335  9286 consensus_queue.cc:1048] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:32.989378  9088 raft_consensus.cc:1275] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 97b41bf0005c4e9c8a9ed4795fc9080f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:32.994046  9286 consensus_queue.cc:1048] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:35.319761  8985 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37740:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:28:35.339188  8985 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37740:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:28:35.380622 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:28:35.517356  9468 log.cc:927] Time spent T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:28:35.629112  9485 tablet.cc:1724] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:35.664902  9128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:38619, user_credentials={real_user=slave}} blocked reactor thread for 45555.7us
I20260709 12:28:35.695452  9087 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9a991ef86b041acb942f1842ccbe254"
dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
mode: GRACEFUL
new_leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:52972
I20260709 12:28:35.696538  9087 raft_consensus.cc:606] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Received request to transfer leadership to TS 196d3384261d4a8dbf0a91f553b1ef4a
W20260709 12:28:35.702642  9483 log.cc:927] Time spent T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:28:35.643518  9493 tablet.cc:1724] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:35.729107  9484 log.cc:927] Time spent T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:28:35.718061  9116 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:98bec205679d4737b1136c2050637373)
W20260709 12:28:35.740729  9116 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:98bec205679d4737b1136c2050637373)
I20260709 12:28:35.744365  9502 tablet.cc:1724] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:35.750895  9500 tablet.cc:1724] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:35.764911  9510 tablet.cc:1724] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:35.799094  9506 tablet.cc:1724] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:35.914194  9265 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:2da3589398f44c809c760886d4f1879e)
W20260709 12:28:36.055681  9202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.66:44727, user_credentials={real_user=slave}} blocked reactor thread for 146788us
I20260709 12:28:36.079022  9514 tablet.cc:1724] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.082276  9510 tablet.cc:1724] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.083825  9502 tablet.cc:1724] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:36.085238  9476 log.cc:927] Time spent T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a: Append to log took a long time: real 0.398s	user 0.000s	sys 0.003s
I20260709 12:28:36.105639  9516 tablet.cc:1724] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:36.150298  9507 log.cc:927] Time spent T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a: Append to log took a long time: real 0.093s	user 0.000s	sys 0.003s
I20260709 12:28:36.155303  9500 tablet.cc:1724] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.176103  9447 raft_consensus.cc:993] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: : Instructing follower 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:36.176725  9447 raft_consensus.cc:1081] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Signalling peer 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:36.192312  9519 tablet.cc:1724] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.230719  9500 tablet.cc:1724] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.233466  9514 tablet.cc:1724] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:36.238317  9053 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.67:46713, user_credentials={real_user=slave}} blocked reactor thread for 43665.2us
I20260709 12:28:36.244886  9523 tablet.cc:1724] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.246837  9236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9a991ef86b041acb942f1842ccbe254"
dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:39004
I20260709 12:28:36.247584  9236 raft_consensus.cc:493] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:36.248076  9236 raft_consensus.cc:3060] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.252022  9510 tablet.cc:1724] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.255338  9520 tablet.cc:1724] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.259337  9523 tablet.cc:1724] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.274683  9514 tablet.cc:1724] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.280251  9500 tablet.cc:1724] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.282162  9516 tablet.cc:1724] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.283524  9485 tablet.cc:1724] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.285735  9506 tablet.cc:1724] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.303378  9493 tablet.cc:1724] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.304018  9502 tablet.cc:1724] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.330961  9493 tablet.cc:1724] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.442421  9236 raft_consensus.cc:515] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.445153  9236 leader_election.cc:290] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:36.479219  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9a991ef86b041acb942f1842ccbe254" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:36.479952  9089 raft_consensus.cc:3055] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:36.480289  9089 raft_consensus.cc:740] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:36.481251  9089 consensus_queue.cc:260] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
W20260709 12:28:36.484192  9497 log.cc:927] Time spent T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: Append to log took a long time: real 0.800s	user 0.000s	sys 0.006s
I20260709 12:28:36.486720  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9a991ef86b041acb942f1842ccbe254" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:36.487372  9163 raft_consensus.cc:3060] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.487994  9500 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:58176 (request call id 19) took 1132 ms. Trace:
I20260709 12:28:36.488430  9500 rpcz_store.cc:276] 0709 12:28:35.356008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.356192 (+   184us) service_pool.cc:224] Handling call
0709 12:28:36.487916 (+1131724us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:35.415797 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 12:28:35.427905 (+ 12108us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 12:28:35.427938 (+    33us) alter_schema_op.cc:66] Acquired schema lock
0709 12:28:35.428003 (+    65us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 12:28:35.428097 (+    94us) alter_schema_op.cc:128] START. Timestamp: 1783600115428044
0709 12:28:35.428121 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:28:35.444173 (+ 16052us) log.cc:844] Serialized 204 byte log entry
0709 12:28:36.264586 (+820413us) op_driver.cc:464] REPLICATION: finished
0709 12:28:36.280075 (+ 15489us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 12:28:36.303423 (+ 23348us) log.cc:844] Serialized 5 byte log entry
0709 12:28:36.486954 (+183531us) tablet_metadata.cc:665] Metadata flushed
0709 12:28:36.487687 (+   733us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.487767 (+    80us) alter_schema_op.cc:73] Released schema lock
0709 12:28:36.487785 (+    18us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":15253,"mutex_wait_us":29826,"prepare.queue_time_us":53329,"prepare.run_cpu_time_us":24739,"prepare.run_wall_time_us":207352,"raft.queue_time_us":40017,"raft.run_cpu_time_us":2091,"raft.run_wall_time_us":92172,"replication_time_us":836258,"thread_start_us":152284,"threads_started":3,"wal-append.queue_time_us":107218}]]}
I20260709 12:28:36.494601  9500 tablet.cc:1724] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:36.493062  9089 raft_consensus.cc:3060] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.498186  9163 raft_consensus.cc:2468] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 2.
I20260709 12:28:36.499625  9202 leader_election.cc:304] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:36.502597  9518 raft_consensus.cc:2804] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:36.503115  9518 raft_consensus.cc:697] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
W20260709 12:28:36.504510  9128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.29.37.65:38619, user_credentials={real_user=slave}} blocked reactor thread for 208935us
I20260709 12:28:36.504169  9518 consensus_queue.cc:237] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.504230  9089 raft_consensus.cc:2468] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 2.
I20260709 12:28:36.524439  9163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e08151a3778945128367bc763a6f13ad"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
mode: GRACEFUL
new_leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:36798
I20260709 12:28:36.529366  9163 raft_consensus.cc:606] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Received request to transfer leadership to TS 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:36.541834 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:28:36.543157 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:36.555832  8987 catalog_manager.cc:5697] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 1 to 2, leader changed from 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65) to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 2 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:36.569056  9089 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa"
dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
mode: GRACEFUL
new_leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:52972
I20260709 12:28:36.569694  9089 raft_consensus.cc:606] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Received request to transfer leadership to TS 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:36.573032  9163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646"
dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
mode: GRACEFUL
new_leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:36798
I20260709 12:28:36.573566  9163 raft_consensus.cc:606] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Received request to transfer leadership to TS 196d3384261d4a8dbf0a91f553b1ef4a
I20260709 12:28:36.577073 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:36.577749 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:36.590590  8988 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731: soft_deleted_table [id=21d763085b5b482fbe445a3d11a4bceb] alter complete (version 1)
I20260709 12:28:36.695240  9526 raft_consensus.cc:993] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f: : Instructing follower 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:36.695772  9526 raft_consensus.cc:1081] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Signalling peer 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:36.697391  9236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646"
dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:38990
I20260709 12:28:36.697976  9236 raft_consensus.cc:493] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:36.698364  9236 raft_consensus.cc:3060] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.704665  9236 raft_consensus.cc:515] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.706631  9236 leader_election.cc:290] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:36.707297  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:36.707978  9089 raft_consensus.cc:3060] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.709460  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f1ca365e1da48cebf49533db981e646" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:36.710071  9163 raft_consensus.cc:3055] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:36.710412  9163 raft_consensus.cc:740] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:36.711234  9163 consensus_queue.cc:260] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.712548  9163 raft_consensus.cc:3060] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.715341  9089 raft_consensus.cc:2468] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 2.
I20260709 12:28:36.716730  9202 leader_election.cc:304] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:36.717586  9518 raft_consensus.cc:2804] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:36.718026  9518 raft_consensus.cc:697] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:36.719274  9163 raft_consensus.cc:2468] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 2.
I20260709 12:28:36.718935  9518 consensus_queue.cc:237] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.732604  8987 catalog_manager.cc:5697] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 1 to 2, leader changed from 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66) to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 2 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:36.742394  9524 raft_consensus.cc:993] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5: : Instructing follower 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:36.742861  9524 raft_consensus.cc:1081] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Signalling peer 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:36.745666  9232 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa"
dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:39004
I20260709 12:28:36.746417  9232 raft_consensus.cc:493] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:36.746934  9232 raft_consensus.cc:3060] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.755976  9232 raft_consensus.cc:515] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.758139  9232 leader_election.cc:290] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:36.758786  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:36.759148  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12f2b301f43d4850acaee49d3df55afa" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:36.759426  9089 raft_consensus.cc:3055] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:36.759747  9163 raft_consensus.cc:3060] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.759783  9089 raft_consensus.cc:740] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f93ba6f3ac741ee93fa5765a94326e5, State: Running, Role: LEADER
I20260709 12:28:36.760710  9089 consensus_queue.cc:260] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.762066  9089 raft_consensus.cc:3060] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:36.766683  9163 raft_consensus.cc:2468] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 2.
I20260709 12:28:36.767963  9202 leader_election.cc:304] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 97b41bf0005c4e9c8a9ed4795fc9080f; no voters: 
I20260709 12:28:36.768849  9518 raft_consensus.cc:2804] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:36.769017  9089 raft_consensus.cc:2468] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 2.
I20260709 12:28:36.769657  9518 raft_consensus.cc:697] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:36.770669  9518 consensus_queue.cc:237] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.780193  8987 catalog_manager.cc:5697] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 1 to 2, leader changed from 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65) to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 2 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:36.802388  9490 raft_consensus.cc:993] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f: : Instructing follower 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:36.802847  9526 raft_consensus.cc:1081] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Signalling peer 196d3384261d4a8dbf0a91f553b1ef4a to start an election
I20260709 12:28:36.804222  9236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e08151a3778945128367bc763a6f13ad"
dest_uuid: "196d3384261d4a8dbf0a91f553b1ef4a"
 from {username='slave'} at 127.0.0.1:38990
I20260709 12:28:36.804700  9236 raft_consensus.cc:493] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:36.805018  9236 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:36.810668  9236 raft_consensus.cc:515] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.812673  9236 leader_election.cc:290] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 election: Requested vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:36.813295  9089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08151a3778945128367bc763a6f13ad" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "3f93ba6f3ac741ee93fa5765a94326e5"
I20260709 12:28:36.813688  9163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08151a3778945128367bc763a6f13ad" candidate_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f"
I20260709 12:28:36.814025  9089 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:36.814378  9163 raft_consensus.cc:3055] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:36.814631  9163 raft_consensus.cc:740] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 97b41bf0005c4e9c8a9ed4795fc9080f, State: Running, Role: LEADER
I20260709 12:28:36.815240  9163 consensus_queue.cc:260] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.816151  9163 raft_consensus.cc:3060] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:36.819790  9089 raft_consensus.cc:2468] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 3.
I20260709 12:28:36.820560  9163 raft_consensus.cc:2468] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 196d3384261d4a8dbf0a91f553b1ef4a in term 3.
I20260709 12:28:36.820820  9202 leader_election.cc:304] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a, 3f93ba6f3ac741ee93fa5765a94326e5; no voters: 
I20260709 12:28:36.821638  9518 raft_consensus.cc:2804] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:36.822146  9518 raft_consensus.cc:697] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 3 LEADER]: Becoming Leader. State: Replica: 196d3384261d4a8dbf0a91f553b1ef4a, State: Running, Role: LEADER
I20260709 12:28:36.822929  9518 consensus_queue.cc:237] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:36.829488  8987 catalog_manager.cc:5697] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a reported cstate change: term changed from 2 to 3, leader changed from 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66) to 196d3384261d4a8dbf0a91f553b1ef4a (127.29.37.67). New cstate: current_term: 3 leader_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:36.952560  9089 raft_consensus.cc:1275] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:28:36.953747  9518 consensus_queue.cc:1048] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:36.961318  9163 raft_consensus.cc:1275] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:28:36.962771  9446 consensus_queue.cc:1048] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:37.208346  9163 raft_consensus.cc:1275] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 3 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:28:37.209837  9446 consensus_queue.cc:1048] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:37.218319  9089 raft_consensus.cc:1275] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 3 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:28:37.220063  9538 consensus_queue.cc:1048] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:37.255980  9163 raft_consensus.cc:1275] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:37.257269  9446 consensus_queue.cc:1048] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:37.258395  9089 raft_consensus.cc:1275] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:37.259706  9446 consensus_queue.cc:1048] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:37.275470  9089 raft_consensus.cc:1275] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:37.276682  9164 raft_consensus.cc:1275] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Refusing update from remote peer 196d3384261d4a8dbf0a91f553b1ef4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:37.280586  9446 consensus_queue.cc:1048] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:37.282101  9446 consensus_queue.cc:1048] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:37.342537  9111 debug-util.cc:398] Leaking SignalData structure 0x7b080042c2e0 after lost signal to thread 29848
W20260709 12:28:37.343611  9111 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6020 after lost signal to thread 9022
W20260709 12:28:37.344970  9111 debug-util.cc:398] Leaking SignalData structure 0x7b08002fa9e0 after lost signal to thread 9114
W20260709 12:28:37.346097  9111 debug-util.cc:398] Leaking SignalData structure 0x7b0800214140 after lost signal to thread 9189
W20260709 12:28:37.347282  9111 debug-util.cc:398] Leaking SignalData structure 0x7b080032c060 after lost signal to thread 9263
I20260709 12:28:37.579177 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:37.587236 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:37.587803 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:37.595577 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:37.596084 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:38.597216 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:38.605288 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:38.605850 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:38.613554 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:38.614740 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:39.615935 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:39.624143 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:39.624694 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:39.629969 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:39.630398 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:40.631419 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:40.639269 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:40.639850 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:40.644672 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:40.645035 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:41.646152 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:41.655602 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:41.656206 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:41.663879 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:41.664388 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:42.665608 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:42.674820 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:42.675489 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:42.683547 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:42.684077 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:43.685209 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:43.691732 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:43.692303 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:43.698159 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:43.698704 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:44.699894 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:44.706804 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:44.707440 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:44.715469 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:44.716007 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:45.717515 29845 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:45.727093 29845 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:45.727696 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:45.737653 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:45.738272 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:28:46.928714  9186 debug-util.cc:398] Leaking SignalData structure 0x7b080017b7c0 after lost signal to thread 29848
W20260709 12:28:46.930258  9186 debug-util.cc:398] Leaking SignalData structure 0x7b0800063940 after lost signal to thread 9022
W20260709 12:28:46.931677  9186 debug-util.cc:398] Leaking SignalData structure 0x7b08002ba7c0 after lost signal to thread 9114
W20260709 12:28:46.932904  9186 debug-util.cc:398] Leaking SignalData structure 0x7b0800129460 after lost signal to thread 9189
W20260709 12:28:46.934139  9186 debug-util.cc:398] Leaking SignalData structure 0x7b08000f8d20 after lost signal to thread 9263
I20260709 12:28:46.988322 29845 tablet_server.cc:179] TabletServer@127.29.37.65:0 shutting down...
W20260709 12:28:47.016009  9128 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
W20260709 12:28:47.028183  9128 consensus_peers.cc:597] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.036444  9128 consensus_peers.cc:597] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.056032  9202 consensus_peers.cc:597] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.057353 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:28:47.058496  9202 consensus_peers.cc:597] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.058692 29845 tablet_replica.cc:333] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.060312 29845 raft_consensus.cc:2243] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.061401 29845 raft_consensus.cc:2272] T 47012003b93e4a039b6747b9d3826856 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.065492 29845 tablet_replica.cc:333] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.066447 29845 raft_consensus.cc:2243] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.067830 29845 raft_consensus.cc:2272] T 57de5299d7574277a4f32823f094feb2 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.071951 29845 tablet_replica.cc:333] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.073047 29845 raft_consensus.cc:2243] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.073725 29845 raft_consensus.cc:2272] T e08151a3778945128367bc763a6f13ad P 3f93ba6f3ac741ee93fa5765a94326e5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.077545 29845 tablet_replica.cc:333] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.078527 29845 raft_consensus.cc:2243] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.079282 29845 raft_consensus.cc:2272] T d9a991ef86b041acb942f1842ccbe254 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.083869 29845 tablet_replica.cc:333] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.084879 29845 raft_consensus.cc:2243] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.086305 29845 raft_consensus.cc:2272] T 75aac328039c40c08f8a13b0b78c6307 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.090289 29845 tablet_replica.cc:333] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.091158 29845 raft_consensus.cc:2243] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.092530 29845 raft_consensus.cc:2272] T e2920b7e69b24a33bde09c66cd1a8db9 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.095988 29845 tablet_replica.cc:333] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.096962 29845 raft_consensus.cc:2243] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.097676 29845 raft_consensus.cc:2272] T 12f2b301f43d4850acaee49d3df55afa P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.101603 29845 tablet_replica.cc:333] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.102514 29845 raft_consensus.cc:2243] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:47.103449  9202 consensus_peers.cc:597] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.103509 29845 raft_consensus.cc:2272] T e7cb21b02d9b4b3bb1d156baca45ec48 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.107455 29845 tablet_replica.cc:333] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.108386 29845 raft_consensus.cc:2243] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.109153 29845 raft_consensus.cc:2272] T 0c01405fa79f45278781c477b429c667 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.113071 29845 tablet_replica.cc:333] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.113929 29845 raft_consensus.cc:2243] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.114672 29845 raft_consensus.cc:2272] T 92ad52027db5443b9e53187cfcbbf9e3 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.118273 29845 tablet_replica.cc:333] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.119145 29845 raft_consensus.cc:2243] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.119760 29845 raft_consensus.cc:2272] T 20ecb85c443e42c197335b7b69b97eff P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.123242 29845 tablet_replica.cc:333] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.124184 29845 raft_consensus.cc:2243] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.124935 29845 raft_consensus.cc:2272] T 257bddd4b4564ccc9b17f16c85d7fc39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.128614 29845 tablet_replica.cc:333] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.129446 29845 raft_consensus.cc:2243] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.130954 29845 raft_consensus.cc:2272] T 98bec205679d4737b1136c2050637373 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.134681 29845 tablet_replica.cc:333] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.135483 29845 raft_consensus.cc:2243] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.136209 29845 raft_consensus.cc:2272] T 04120eea993a4a4c948f77e924c1cd39 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.139897 29845 tablet_replica.cc:333] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.140657 29845 raft_consensus.cc:2243] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.141371 29845 raft_consensus.cc:2272] T 2da3589398f44c809c760886d4f1879e P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.144703 29845 tablet_replica.cc:333] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.145445 29845 raft_consensus.cc:2243] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.146989 29845 raft_consensus.cc:2272] T 2a77ba1e532f485abf98c54769535556 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.150364 29845 tablet_replica.cc:333] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.151115 29845 raft_consensus.cc:2243] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.152346 29845 raft_consensus.cc:2272] T 68339bccb9ab48bf8a795f846dd75247 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.156059 29845 tablet_replica.cc:333] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5: stopping tablet replica
I20260709 12:28:47.156795 29845 raft_consensus.cc:2243] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.157564 29845 raft_consensus.cc:2272] T 7f1ca365e1da48cebf49533db981e646 P 3f93ba6f3ac741ee93fa5765a94326e5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.202402  9202 consensus_peers.cc:597] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.226085  9128 consensus_peers.cc:597] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.247319  9128 consensus_peers.cc:597] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.255095 29845 tablet_server.cc:196] TabletServer@127.29.37.65:0 shutdown complete.
W20260709 12:28:47.271737  9128 consensus_peers.cc:597] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.281944  9128 consensus_peers.cc:597] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.329070 29845 tablet_server.cc:179] TabletServer@127.29.37.66:0 shutting down...
I20260709 12:28:47.371693 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:47.372663 29845 tablet_replica.cc:333] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.373921 29845 raft_consensus.cc:2243] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.374734 29845 raft_consensus.cc:2272] T 75aac328039c40c08f8a13b0b78c6307 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.377499 29845 tablet_replica.cc:333] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.378181 29845 raft_consensus.cc:2243] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.378885 29845 raft_consensus.cc:2272] T 47012003b93e4a039b6747b9d3826856 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.381448 29845 tablet_replica.cc:333] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.382174 29845 raft_consensus.cc:2243] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.382750 29845 raft_consensus.cc:2272] T e08151a3778945128367bc763a6f13ad P 97b41bf0005c4e9c8a9ed4795fc9080f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.385577 29845 tablet_replica.cc:333] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.386394 29845 raft_consensus.cc:2243] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.387151 29845 raft_consensus.cc:2272] T d9a991ef86b041acb942f1842ccbe254 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.390841 29845 tablet_replica.cc:333] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.391537 29845 raft_consensus.cc:2243] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.392166 29845 raft_consensus.cc:2272] T 57de5299d7574277a4f32823f094feb2 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.394922 29845 tablet_replica.cc:333] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.395582 29845 raft_consensus.cc:2243] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.396149 29845 raft_consensus.cc:2272] T 2a77ba1e532f485abf98c54769535556 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.398674 29845 tablet_replica.cc:333] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.399220 29845 raft_consensus.cc:2243] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.400187 29845 raft_consensus.cc:2272] T 2da3589398f44c809c760886d4f1879e P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.402720 29845 tablet_replica.cc:333] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.403426 29845 raft_consensus.cc:2243] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.404003 29845 raft_consensus.cc:2272] T e2920b7e69b24a33bde09c66cd1a8db9 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.406606 29845 tablet_replica.cc:333] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.407148 29845 raft_consensus.cc:2243] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.407761 29845 raft_consensus.cc:2272] T 68339bccb9ab48bf8a795f846dd75247 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.410930 29845 tablet_replica.cc:333] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.411708 29845 raft_consensus.cc:2243] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:28:47.412432  9202 consensus_peers.cc:597] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Couldn't send request to peer 97b41bf0005c4e9c8a9ed4795fc9080f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.412907 29845 raft_consensus.cc:2272] T e7cb21b02d9b4b3bb1d156baca45ec48 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.415692 29845 tablet_replica.cc:333] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.416329 29845 raft_consensus.cc:2243] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.416867 29845 raft_consensus.cc:2272] T 12f2b301f43d4850acaee49d3df55afa P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.419874 29845 tablet_replica.cc:333] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.420475 29845 raft_consensus.cc:2243] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.421562 29845 raft_consensus.cc:2272] T 20ecb85c443e42c197335b7b69b97eff P 97b41bf0005c4e9c8a9ed4795fc9080f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.424187 29845 tablet_replica.cc:333] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.424764 29845 raft_consensus.cc:2243] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.425788 29845 raft_consensus.cc:2272] T 0c01405fa79f45278781c477b429c667 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.428318 29845 tablet_replica.cc:333] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.428890 29845 raft_consensus.cc:2243] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.430025 29845 raft_consensus.cc:2272] T 92ad52027db5443b9e53187cfcbbf9e3 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.432760 29845 tablet_replica.cc:333] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.433322 29845 raft_consensus.cc:2243] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.433979 29845 raft_consensus.cc:2272] T 98bec205679d4737b1136c2050637373 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.436518 29845 tablet_replica.cc:333] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.437057 29845 raft_consensus.cc:2243] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:47.437983 29845 raft_consensus.cc:2272] T 257bddd4b4564ccc9b17f16c85d7fc39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.440649 29845 tablet_replica.cc:333] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.441192 29845 raft_consensus.cc:2243] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.441752 29845 raft_consensus.cc:2272] T 04120eea993a4a4c948f77e924c1cd39 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:47.444532 29845 tablet_replica.cc:333] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f: stopping tablet replica
I20260709 12:28:47.445032 29845 raft_consensus.cc:2243] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:47.445587 29845 raft_consensus.cc:2272] T 7f1ca365e1da48cebf49533db981e646 P 97b41bf0005c4e9c8a9ed4795fc9080f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:47.480000  9202 consensus_peers.cc:597] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.509048  9202 consensus_peers.cc:597] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.577661  9202 consensus_peers.cc:597] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Couldn't send request to peer 97b41bf0005c4e9c8a9ed4795fc9080f. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.620059  9202 consensus_peers.cc:597] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Couldn't send request to peer 97b41bf0005c4e9c8a9ed4795fc9080f. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.676527  9202 consensus_peers.cc:597] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Couldn't send request to peer 97b41bf0005c4e9c8a9ed4795fc9080f. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.769310  9202 consensus_peers.cc:597] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Couldn't send request to peer 97b41bf0005c4e9c8a9ed4795fc9080f. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:47.893852  9202 consensus_peers.cc:597] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Couldn't send request to peer 97b41bf0005c4e9c8a9ed4795fc9080f. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.004199  9571 raft_consensus.cc:493] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f93ba6f3ac741ee93fa5765a94326e5)
I20260709 12:28:48.004849  9571 raft_consensus.cc:515] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } }
I20260709 12:28:48.007359  9571 leader_election.cc:290] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.018105  9202 leader_election.cc:336] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.018858  9202 leader_election.cc:336] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.019171  9202 leader_election.cc:304] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.019905  9572 raft_consensus.cc:2749] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:48.452056  9572 raft_consensus.cc:493] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f93ba6f3ac741ee93fa5765a94326e5)
I20260709 12:28:48.452561  9572 raft_consensus.cc:515] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.454356  9572 leader_election.cc:290] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:48.458185  9572 raft_consensus.cc:493] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f93ba6f3ac741ee93fa5765a94326e5)
W20260709 12:28:48.458429  9202 leader_election.cc:336] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.459498  9202 leader_election.cc:336] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.458868  9572 raft_consensus.cc:515] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.459512  9570 raft_consensus.cc:493] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 97b41bf0005c4e9c8a9ed4795fc9080f)
I20260709 12:28:48.460148  9202 leader_election.cc:304] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.460345  9570 raft_consensus.cc:515] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.461663  9572 leader_election.cc:290] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:48.462145  9572 raft_consensus.cc:2749] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:48.462620  9570 leader_election.cc:290] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
I20260709 12:28:48.463325  9570 raft_consensus.cc:493] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f93ba6f3ac741ee93fa5765a94326e5)
I20260709 12:28:48.463780  9570 raft_consensus.cc:515] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.466367  9570 leader_election.cc:290] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.468058  9202 leader_election.cc:336] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.468581  9202 leader_election.cc:336] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.469359  9202 leader_election.cc:336] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.469740  9202 leader_election.cc:304] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.470342  9570 raft_consensus.cc:2749] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:28:48.470499  9202 leader_election.cc:336] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.471067  9202 leader_election.cc:304] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.471621  9570 raft_consensus.cc:2749] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:28:48.473538  9202 leader_election.cc:336] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.474159  9202 leader_election.cc:336] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.474516  9202 leader_election.cc:304] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.475029  9570 raft_consensus.cc:2749] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:28:48.483625 29845 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 9186)
I20260709 12:28:48.519775  9570 raft_consensus.cc:493] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f93ba6f3ac741ee93fa5765a94326e5)
I20260709 12:28:48.520399  9570 raft_consensus.cc:515] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.522365  9570 leader_election.cc:290] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.526717  9202 leader_election.cc:336] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.527403  9202 leader_election.cc:336] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.527786  9202 leader_election.cc:304] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.528455  9570 raft_consensus.cc:2749] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:48.552316  9570 raft_consensus.cc:493] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 97b41bf0005c4e9c8a9ed4795fc9080f)
I20260709 12:28:48.552922  9570 raft_consensus.cc:515] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.555382  9570 leader_election.cc:290] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.559342  9202 leader_election.cc:336] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.560259  9202 leader_election.cc:336] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.560626  9202 leader_election.cc:304] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.561280  9570 raft_consensus.cc:2749] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:48.570294  9570 raft_consensus.cc:493] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 97b41bf0005c4e9c8a9ed4795fc9080f)
I20260709 12:28:48.570816  9570 raft_consensus.cc:515] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.572992  9570 leader_election.cc:290] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.576663  9202 leader_election.cc:336] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.577629  9202 leader_election.cc:336] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.577940  9202 leader_election.cc:304] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.578531  9570 raft_consensus.cc:2749] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:48.583439  9570 raft_consensus.cc:493] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3f93ba6f3ac741ee93fa5765a94326e5)
I20260709 12:28:48.583952  9570 raft_consensus.cc:515] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.586187  9570 leader_election.cc:290] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.590452  9202 leader_election.cc:336] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.591053  9202 leader_election.cc:336] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.591436  9202 leader_election.cc:304] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.592089  9570 raft_consensus.cc:2749] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:48.602954  9570 raft_consensus.cc:493] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 97b41bf0005c4e9c8a9ed4795fc9080f)
I20260709 12:28:48.603374  9570 raft_consensus.cc:515] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.604938  9570 leader_election.cc:290] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.608675  9202 leader_election.cc:336] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.609335  9202 leader_election.cc:336] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.609715  9202 leader_election.cc:304] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.610400  9570 raft_consensus.cc:2749] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:48.853188  9570 raft_consensus.cc:493] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 97b41bf0005c4e9c8a9ed4795fc9080f)
I20260709 12:28:48.853693  9570 raft_consensus.cc:515] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.855583  9570 leader_election.cc:290] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.859272  9202 leader_election.cc:336] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.860035  9202 leader_election.cc:336] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.860358  9202 leader_election.cc:304] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.860920  9570 raft_consensus.cc:2749] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:48.890049  9570 raft_consensus.cc:493] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 97b41bf0005c4e9c8a9ed4795fc9080f)
I20260709 12:28:48.890552  9570 raft_consensus.cc:515] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f93ba6f3ac741ee93fa5765a94326e5" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 38619 } } peers { permanent_uuid: "97b41bf0005c4e9c8a9ed4795fc9080f" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 44727 } } peers { permanent_uuid: "196d3384261d4a8dbf0a91f553b1ef4a" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 46713 } }
I20260709 12:28:48.892362  9570 leader_election.cc:290] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619), 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727)
W20260709 12:28:48.895910  9202 leader_election.cc:336] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111)
W20260709 12:28:48.896628  9202 leader_election.cc:336] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111)
I20260709 12:28:48.896955  9202 leader_election.cc:304] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 196d3384261d4a8dbf0a91f553b1ef4a; no voters: 3f93ba6f3ac741ee93fa5765a94326e5, 97b41bf0005c4e9c8a9ed4795fc9080f
I20260709 12:28:48.897508  9570 raft_consensus.cc:2749] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:28:49.466646  9202 consensus_peers.cc:597] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:49.484025 29845 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 9186)
W20260709 12:28:49.519814  9202 consensus_peers.cc:597] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:49.656934  9202 consensus_peers.cc:597] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:49.807909  9202 consensus_peers.cc:597] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:49.861017 29845 tablet_server.cc:196] TabletServer@127.29.37.66:0 shutdown complete.
I20260709 12:28:49.903478 29845 tablet_server.cc:179] TabletServer@127.29.37.67:0 shutting down...
W20260709 12:28:49.933969  9202 consensus_peers.cc:597] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 97b41bf0005c4e9c8a9ed4795fc9080f (127.29.37.66:44727): Couldn't send request to peer 97b41bf0005c4e9c8a9ed4795fc9080f. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:44727: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:49.942567 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:49.943361 29845 tablet_replica.cc:333] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.944054 29845 raft_consensus.cc:2243] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.944589 29845 raft_consensus.cc:2272] T 57de5299d7574277a4f32823f094feb2 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.947000 29845 tablet_replica.cc:333] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.947599 29845 raft_consensus.cc:2243] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.948417 29845 raft_consensus.cc:2272] T 47012003b93e4a039b6747b9d3826856 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.950519 29845 tablet_replica.cc:333] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.951097 29845 raft_consensus.cc:2243] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.951601 29845 raft_consensus.cc:2272] T 75aac328039c40c08f8a13b0b78c6307 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.953574 29845 tablet_replica.cc:333] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.954145 29845 raft_consensus.cc:2243] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.954668 29845 raft_consensus.cc:2272] T 98bec205679d4737b1136c2050637373 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.956543 29845 tablet_replica.cc:333] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.957108 29845 raft_consensus.cc:2243] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.957597 29845 raft_consensus.cc:2272] T 257bddd4b4564ccc9b17f16c85d7fc39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.959560 29845 tablet_replica.cc:333] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.960100 29845 raft_consensus.cc:2243] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.960703 29845 raft_consensus.cc:2272] T 2a77ba1e532f485abf98c54769535556 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.963600 29845 tablet_replica.cc:333] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.964375 29845 raft_consensus.cc:2243] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:49.964695  9202 consensus_peers.cc:597] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a -> Peer 3f93ba6f3ac741ee93fa5765a94326e5 (127.29.37.65:38619): Couldn't send request to peer 3f93ba6f3ac741ee93fa5765a94326e5. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:38619: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:49.964939 29845 raft_consensus.cc:2272] T 2da3589398f44c809c760886d4f1879e P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.967312 29845 tablet_replica.cc:333] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.967928 29845 raft_consensus.cc:2243] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.968725 29845 raft_consensus.cc:2272] T e08151a3778945128367bc763a6f13ad P 196d3384261d4a8dbf0a91f553b1ef4a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.970844 29845 tablet_replica.cc:333] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.971362 29845 raft_consensus.cc:2243] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.972242 29845 raft_consensus.cc:2272] T d9a991ef86b041acb942f1842ccbe254 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.974267 29845 tablet_replica.cc:333] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.974807 29845 raft_consensus.cc:2243] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.975554 29845 raft_consensus.cc:2272] T 7f1ca365e1da48cebf49533db981e646 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.977480 29845 tablet_replica.cc:333] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.977994 29845 raft_consensus.cc:2243] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.978469 29845 raft_consensus.cc:2272] T 20ecb85c443e42c197335b7b69b97eff P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.980645 29845 tablet_replica.cc:333] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.981199 29845 raft_consensus.cc:2243] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.981645 29845 raft_consensus.cc:2272] T 0c01405fa79f45278781c477b429c667 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.983718 29845 tablet_replica.cc:333] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.984256 29845 raft_consensus.cc:2243] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.984751 29845 raft_consensus.cc:2272] T 92ad52027db5443b9e53187cfcbbf9e3 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.987205 29845 tablet_replica.cc:333] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.987748 29845 raft_consensus.cc:2243] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.988193 29845 raft_consensus.cc:2272] T 68339bccb9ab48bf8a795f846dd75247 P 196d3384261d4a8dbf0a91f553b1ef4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.990454 29845 tablet_replica.cc:333] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.990960 29845 raft_consensus.cc:2243] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.991421 29845 raft_consensus.cc:2272] T e2920b7e69b24a33bde09c66cd1a8db9 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.993502 29845 tablet_replica.cc:333] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.994031 29845 raft_consensus.cc:2243] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:49.994889 29845 raft_consensus.cc:2272] T 04120eea993a4a4c948f77e924c1cd39 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.996980 29845 tablet_replica.cc:333] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:49.997490 29845 raft_consensus.cc:2243] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:49.997963 29845 raft_consensus.cc:2272] T e7cb21b02d9b4b3bb1d156baca45ec48 P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:49.999998 29845 tablet_replica.cc:333] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a: stopping tablet replica
I20260709 12:28:50.000442 29845 raft_consensus.cc:2243] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:50.001220 29845 raft_consensus.cc:2272] T 12f2b301f43d4850acaee49d3df55afa P 196d3384261d4a8dbf0a91f553b1ef4a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:50.043129 29845 tablet_server.cc:196] TabletServer@127.29.37.67:0 shutdown complete.
I20260709 12:28:50.080433 29845 master.cc:562] Master@127.29.37.126:33431 shutting down...
I20260709 12:28:50.103762 29845 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:50.104414 29845 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:50.104766 29845 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6c56aae1034141c78d68463d8f5a3731: stopping tablet replica
I20260709 12:28:50.125869 29845 master.cc:584] Master@127.29.37.126:33431 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (24087 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:28:50.180037 29845 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.29.37.126:45671
I20260709 12:28:50.181161 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:50.186646  9576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:50.187330  9577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:50.187969  9579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:50.189121 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:50.190085 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:50.190647 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:50.190812 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600130190796 us; error 0 us; skew 500 ppm
I20260709 12:28:50.191303 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:50.193706 29845 webserver.cc:533] Webserver started at http://127.29.37.126:37677/ using document root <none> and password file <none>
I20260709 12:28:50.194509 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:50.194928 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:50.195218 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:50.196358 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/master-0-root/instance:
uuid: "5f37ea631d584360855eabe75c4ff517"
format_stamp: "Formatted at 2026-07-09 12:28:50 on dist-test-slave-g1m6"
I20260709 12:28:50.200906 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:28:50.204178  9584 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.204893 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:28:50.205186 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/master-0-root
uuid: "5f37ea631d584360855eabe75c4ff517"
format_stamp: "Formatted at 2026-07-09 12:28:50 on dist-test-slave-g1m6"
I20260709 12:28:50.205447 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:50.219551 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:50.220715 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:50.258709 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.126:45671
I20260709 12:28:50.258859  9635 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.126:45671 every 8 connection(s)
I20260709 12:28:50.262811  9636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.273988  9636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517: Bootstrap starting.
I20260709 12:28:50.278932  9636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.283191  9636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517: No bootstrap required, opened a new log
I20260709 12:28:50.285411  9636 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f37ea631d584360855eabe75c4ff517" member_type: VOTER }
I20260709 12:28:50.285807  9636 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.286037  9636 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f37ea631d584360855eabe75c4ff517, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.286705  9636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f37ea631d584360855eabe75c4ff517" member_type: VOTER }
I20260709 12:28:50.287163  9636 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:50.287390  9636 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:50.287647  9636 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.292414  9636 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f37ea631d584360855eabe75c4ff517" member_type: VOTER }
I20260709 12:28:50.292909  9636 leader_election.cc:304] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f37ea631d584360855eabe75c4ff517; no voters: 
I20260709 12:28:50.294011  9636 leader_election.cc:290] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:50.294464  9639 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.295908  9639 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 1 LEADER]: Becoming Leader. State: Replica: 5f37ea631d584360855eabe75c4ff517, State: Running, Role: LEADER
I20260709 12:28:50.296641  9639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f37ea631d584360855eabe75c4ff517" member_type: VOTER }
I20260709 12:28:50.296933  9636 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:50.301906  9641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f37ea631d584360855eabe75c4ff517. Latest consensus state: current_term: 1 leader_uuid: "5f37ea631d584360855eabe75c4ff517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f37ea631d584360855eabe75c4ff517" member_type: VOTER } }
I20260709 12:28:50.302591  9640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f37ea631d584360855eabe75c4ff517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f37ea631d584360855eabe75c4ff517" member_type: VOTER } }
I20260709 12:28:50.302842  9641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:50.303337  9640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:50.305088  9648 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:50.310438  9648 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:50.311973 29845 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:50.319417  9648 catalog_manager.cc:1383] Generated new cluster ID: f062395c4357445f92dfa6a3382197f9
I20260709 12:28:50.319697  9648 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:50.366460  9648 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:50.367792  9648 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:50.381861  9648 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517: Generated new TSK 0
I20260709 12:28:50.382495  9648 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:50.442822 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:50.449008  9657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:50.450277  9658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:50.451985 29845 server_base.cc:1061] running on GCE node
W20260709 12:28:50.452751  9660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:50.453758 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:50.453971 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:50.454149 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600130454129 us; error 0 us; skew 500 ppm
I20260709 12:28:50.454761 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:50.456990 29845 webserver.cc:533] Webserver started at http://127.29.37.65:44611/ using document root <none> and password file <none>
I20260709 12:28:50.457473 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:50.457679 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:50.457938 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:50.459091 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-0-root/instance:
uuid: "05b878c1341641c8b469d8fa1a927143"
format_stamp: "Formatted at 2026-07-09 12:28:50 on dist-test-slave-g1m6"
I20260709 12:28:50.463375 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:50.466536  9665 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.467278 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:28:50.467567 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-0-root
uuid: "05b878c1341641c8b469d8fa1a927143"
format_stamp: "Formatted at 2026-07-09 12:28:50 on dist-test-slave-g1m6"
I20260709 12:28:50.467855 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:50.488345 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:50.489650 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:50.491168 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:50.493374 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:50.493595 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.493836 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:50.494029 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.534545 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.65:41227
I20260709 12:28:50.534672  9727 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.65:41227 every 8 connection(s)
I20260709 12:28:50.539587 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:50.546653  9732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:50.548192  9733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:50.551011  9735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:50.551962 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:50.553018  9728 heartbeater.cc:344] Connected to a master server at 127.29.37.126:45671
I20260709 12:28:50.553468  9728 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:50.553743 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:50.554078 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:50.554323 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600130554304 us; error 0 us; skew 500 ppm
I20260709 12:28:50.554420  9728 heartbeater.cc:507] Master 127.29.37.126:45671 requested a full tablet report, sending...
I20260709 12:28:50.555151 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:50.556962  9601 ts_manager.cc:194] Registered new tserver with Master: 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227)
I20260709 12:28:50.557720 29845 webserver.cc:533] Webserver started at http://127.29.37.66:34415/ using document root <none> and password file <none>
I20260709 12:28:50.558316 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:50.558511 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:50.558795 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:50.559129  9601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39068
I20260709 12:28:50.560019 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-1-root/instance:
uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
format_stamp: "Formatted at 2026-07-09 12:28:50 on dist-test-slave-g1m6"
I20260709 12:28:50.564935 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:28:50.568070  9740 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.568799 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:28:50.569082 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-1-root
uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
format_stamp: "Formatted at 2026-07-09 12:28:50 on dist-test-slave-g1m6"
I20260709 12:28:50.569469 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:50.592407 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:50.593612 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:50.595067 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:50.597411 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:50.597642 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.597896 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:50.598070 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.637710 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.66:32899
I20260709 12:28:50.637830  9802 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.66:32899 every 8 connection(s)
I20260709 12:28:50.642810 29845 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:50.650880  9806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:50.652467  9807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:50.655563  9803 heartbeater.cc:344] Connected to a master server at 127.29.37.126:45671
W20260709 12:28:50.655905  9809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:50.656096  9803 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:50.656745 29845 server_base.cc:1061] running on GCE node
I20260709 12:28:50.657146  9803 heartbeater.cc:507] Master 127.29.37.126:45671 requested a full tablet report, sending...
I20260709 12:28:50.657689 29845 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:50.657966 29845 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:50.658154 29845 hybrid_clock.cc:648] HybridClock initialized: now 1783600130658137 us; error 0 us; skew 500 ppm
I20260709 12:28:50.658962 29845 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:50.659328  9601 ts_manager.cc:194] Registered new tserver with Master: 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:50.660771  9601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39082
I20260709 12:28:50.661760 29845 webserver.cc:533] Webserver started at http://127.29.37.67:39317/ using document root <none> and password file <none>
I20260709 12:28:50.662425 29845 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:50.662657 29845 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:50.662932 29845 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:50.664135 29845 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-2-root/instance:
uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
format_stamp: "Formatted at 2026-07-09 12:28:50 on dist-test-slave-g1m6"
I20260709 12:28:50.668479 29845 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:28:50.672067  9814 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.672874 29845 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:28:50.673192 29845 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-2-root
uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
format_stamp: "Formatted at 2026-07-09 12:28:50 on dist-test-slave-g1m6"
I20260709 12:28:50.673493 29845 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task_2T3EI/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599734091681-29845-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:50.695582 29845 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:50.696828 29845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:50.698450 29845 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:50.700625 29845 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:50.700850 29845 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.701110 29845 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:50.701290 29845 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:50.741153 29845 rpc_server.cc:307] RPC server started. Bound to: 127.29.37.67:40277
I20260709 12:28:50.741283  9876 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.29.37.67:40277 every 8 connection(s)
I20260709 12:28:50.754554  9877 heartbeater.cc:344] Connected to a master server at 127.29.37.126:45671
I20260709 12:28:50.754997  9877 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:50.755676  9877 heartbeater.cc:507] Master 127.29.37.126:45671 requested a full tablet report, sending...
I20260709 12:28:50.757483  9601 ts_manager.cc:194] Registered new tserver with Master: cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:50.757905 29845 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013523071s
I20260709 12:28:50.758960  9601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39084
I20260709 12:28:50.782171  9601 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39090:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:50.784948  9601 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:50.852682  9768 tablet_service.cc:1511] Processing CreateTablet for tablet 343c84975f3847e1a1dfd9af9a5ad009 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:50.854169  9768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343c84975f3847e1a1dfd9af9a5ad009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.854590  9693 tablet_service.cc:1511] Processing CreateTablet for tablet 343c84975f3847e1a1dfd9af9a5ad009 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:50.855984  9693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343c84975f3847e1a1dfd9af9a5ad009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.855762  9691 tablet_service.cc:1511] Processing CreateTablet for tablet d79af30301c2424b9d62b8caac3fed98 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:50.858117  9692 tablet_service.cc:1511] Processing CreateTablet for tablet e6abcf18a4624879b263a46ce7a2d1b7 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:50.859490  9842 tablet_service.cc:1511] Processing CreateTablet for tablet 343c84975f3847e1a1dfd9af9a5ad009 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:50.860862  9842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 343c84975f3847e1a1dfd9af9a5ad009. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.860705  9767 tablet_service.cc:1511] Processing CreateTablet for tablet e6abcf18a4624879b263a46ce7a2d1b7 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:50.862080  9767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6abcf18a4624879b263a46ce7a2d1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.862571  9841 tablet_service.cc:1511] Processing CreateTablet for tablet e6abcf18a4624879b263a46ce7a2d1b7 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:50.859544  9692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6abcf18a4624879b263a46ce7a2d1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.863924  9841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6abcf18a4624879b263a46ce7a2d1b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.870569  9691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d79af30301c2424b9d62b8caac3fed98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.883047  9840 tablet_service.cc:1511] Processing CreateTablet for tablet d79af30301c2424b9d62b8caac3fed98 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:50.884505  9840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d79af30301c2424b9d62b8caac3fed98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.886933  9766 tablet_service.cc:1511] Processing CreateTablet for tablet d79af30301c2424b9d62b8caac3fed98 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:50.888310  9766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d79af30301c2424b9d62b8caac3fed98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.898375  9690 tablet_service.cc:1511] Processing CreateTablet for tablet 5c12f2b7ab884baeb47afe3912c5e3e6 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:50.899837  9690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c12f2b7ab884baeb47afe3912c5e3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.906522  9765 tablet_service.cc:1511] Processing CreateTablet for tablet 5c12f2b7ab884baeb47afe3912c5e3e6 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:50.907994  9765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c12f2b7ab884baeb47afe3912c5e3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.908802  9894 tablet_bootstrap.cc:492] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:50.913414  9689 tablet_service.cc:1511] Processing CreateTablet for tablet 55eb762448eb4e04b202d365b4212711 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:50.914899  9689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55eb762448eb4e04b202d365b4212711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.923679  9841 tablet_service.cc:1511] Processing CreateTablet for tablet 5c12f2b7ab884baeb47afe3912c5e3e6 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:50.925498  9841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c12f2b7ab884baeb47afe3912c5e3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.934486  9768 tablet_service.cc:1511] Processing CreateTablet for tablet 55eb762448eb4e04b202d365b4212711 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:50.936322  9768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55eb762448eb4e04b202d365b4212711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.954138  9840 tablet_service.cc:1511] Processing CreateTablet for tablet 55eb762448eb4e04b202d365b4212711 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:50.955638  9840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55eb762448eb4e04b202d365b4212711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.955760  9893 tablet_bootstrap.cc:492] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:50.961349  9690 tablet_service.cc:1511] Processing CreateTablet for tablet 93627256cc2d4c94bfabcc2508dce601 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:50.962687  9892 tablet_bootstrap.cc:492] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:50.966598  9690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93627256cc2d4c94bfabcc2508dce601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.972373  9892 tablet_bootstrap.cc:654] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.964963  9893 tablet_bootstrap.cc:654] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.976384  9767 tablet_service.cc:1511] Processing CreateTablet for tablet 93627256cc2d4c94bfabcc2508dce601 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:50.977198  9842 tablet_service.cc:1511] Processing CreateTablet for tablet 93627256cc2d4c94bfabcc2508dce601 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:50.979189  9842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93627256cc2d4c94bfabcc2508dce601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.980247  9691 tablet_service.cc:1511] Processing CreateTablet for tablet dfc9216de2034f97862627395f7661e0 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:50.981575  9691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc9216de2034f97862627395f7661e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.982617  9894 tablet_bootstrap.cc:654] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.985682  9765 tablet_service.cc:1511] Processing CreateTablet for tablet dfc9216de2034f97862627395f7661e0 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:50.987696  9840 tablet_service.cc:1511] Processing CreateTablet for tablet dfc9216de2034f97862627395f7661e0 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:50.988983  9840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc9216de2034f97862627395f7661e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.991269  9765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc9216de2034f97862627395f7661e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.994814  9767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93627256cc2d4c94bfabcc2508dce601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:50.996718  9693 tablet_service.cc:1511] Processing CreateTablet for tablet 5345b9a348a14191aaddaf426ea23439 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:50.998598  9693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5345b9a348a14191aaddaf426ea23439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:51.001050  9892 tablet_bootstrap.cc:492] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.001581  9892 ts_tablet_manager.cc:1403] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.008s
I20260709 12:28:51.002270  9768 tablet_service.cc:1511] Processing CreateTablet for tablet 5345b9a348a14191aaddaf426ea23439 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:51.003655  9768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5345b9a348a14191aaddaf426ea23439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:51.004702  9892 raft_consensus.cc:359] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.007539  9893 tablet_bootstrap.cc:492] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.008119  9893 ts_tablet_manager.cc:1403] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.013s
I20260709 12:28:51.011148  9893 raft_consensus.cc:359] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.011818  9893 raft_consensus.cc:385] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.012086  9893 raft_consensus.cc:740] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.013062  9893 consensus_queue.cc:260] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.014665  9892 raft_consensus.cc:385] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.015057  9892 raft_consensus.cc:740] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.016078  9892 consensus_queue.cc:260] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.016940  9839 tablet_service.cc:1511] Processing CreateTablet for tablet 5345b9a348a14191aaddaf426ea23439 (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:51.018343  9839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5345b9a348a14191aaddaf426ea23439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:51.023137  9838 tablet_service.cc:1511] Processing CreateTablet for tablet fb936063bf7442549dcb3a3bf3964d5c (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:51.023840  9892 ts_tablet_manager.cc:1434] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.003s
I20260709 12:28:51.024523  9838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb936063bf7442549dcb3a3bf3964d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:51.025059  9728 heartbeater.cc:499] Master 127.29.37.126:45671 was elected leader, sending a full tablet report...
I20260709 12:28:51.029474  9766 tablet_service.cc:1511] Processing CreateTablet for tablet fb936063bf7442549dcb3a3bf3964d5c (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:51.030467  9689 tablet_service.cc:1511] Processing CreateTablet for tablet fb936063bf7442549dcb3a3bf3964d5c (DEFAULT_TABLE table=test-workload [id=d7c6215b745b40fdb008868ae553aee5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:51.031531  9766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb936063bf7442549dcb3a3bf3964d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:51.031745  9689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb936063bf7442549dcb3a3bf3964d5c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:51.044708  9729 tablet.cc:2406] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:28:51.047286  9899 raft_consensus.cc:493] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.047984  9899 raft_consensus.cc:515] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.060286  9899 leader_election.cc:290] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.081252  9877 heartbeater.cc:499] Master 127.29.37.126:45671 was elected leader, sending a full tablet report...
I20260709 12:28:51.082309  9894 tablet_bootstrap.cc:492] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.093600  9894 ts_tablet_manager.cc:1403] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.185s	user 0.024s	sys 0.005s
I20260709 12:28:51.096621  9894 raft_consensus.cc:359] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.097352  9894 raft_consensus.cc:385] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.097645  9894 raft_consensus.cc:740] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.080513  9892 tablet_bootstrap.cc:492] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:51.098531  9894 consensus_queue.cc:260] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.083562  9893 ts_tablet_manager.cc:1434] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.075s	user 0.005s	sys 0.023s
I20260709 12:28:51.103439  9893 tablet_bootstrap.cc:492] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:51.108057  9892 tablet_bootstrap.cc:654] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.127118  9803 heartbeater.cc:499] Master 127.29.37.126:45671 was elected leader, sending a full tablet report...
I20260709 12:28:51.129122  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343c84975f3847e1a1dfd9af9a5ad009" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.118893  9894 ts_tablet_manager.cc:1434] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.012s
I20260709 12:28:51.129909  9778 raft_consensus.cc:2468] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:51.131067  9894 tablet_bootstrap.cc:492] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:51.131553  9669 leader_election.cc:304] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:51.132566  9899 raft_consensus.cc:2804] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.132966  9899 raft_consensus.cc:493] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.133368  9899 raft_consensus.cc:3060] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.138314  9894 tablet_bootstrap.cc:654] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.141997  9893 tablet_bootstrap.cc:654] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.141178  9899 raft_consensus.cc:515] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.145107  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343c84975f3847e1a1dfd9af9a5ad009" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:51.145751  9778 raft_consensus.cc:3060] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.149621  9892 tablet_bootstrap.cc:492] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.150429  9892 ts_tablet_manager.cc:1403] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.093s	user 0.007s	sys 0.007s
I20260709 12:28:51.152310  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343c84975f3847e1a1dfd9af9a5ad009" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:51.153282  9852 raft_consensus.cc:2468] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:51.153867  9894 tablet_bootstrap.cc:492] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.153745  9892 raft_consensus.cc:359] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.154474  9894 ts_tablet_manager.cc:1403] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.008s
I20260709 12:28:51.154589  9892 raft_consensus.cc:385] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.155023  9892 raft_consensus.cc:740] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.155969  9892 consensus_queue.cc:260] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.157620  9894 raft_consensus.cc:359] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.158757  9894 raft_consensus.cc:385] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.159402  9894 raft_consensus.cc:740] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.160646  9894 consensus_queue.cc:260] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.162981  9778 raft_consensus.cc:2468] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:51.163813  9894 ts_tablet_manager.cc:1434] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:28:51.168135  9894 tablet_bootstrap.cc:492] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:51.170172  9893 tablet_bootstrap.cc:492] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.165624  9892 ts_tablet_manager.cc:1434] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:28:51.165822  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343c84975f3847e1a1dfd9af9a5ad009" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:51.166364  9899 leader_election.cc:290] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.170763  9893 ts_tablet_manager.cc:1403] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.068s	user 0.013s	sys 0.018s
I20260709 12:28:51.171173  9852 raft_consensus.cc:3060] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.171801  9669 leader_election.cc:304] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:51.172715  9899 raft_consensus.cc:2804] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.173238  9899 raft_consensus.cc:697] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:51.174136  9899 consensus_queue.cc:237] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.174331  9893 raft_consensus.cc:359] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.175155  9893 raft_consensus.cc:385] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.175531  9893 raft_consensus.cc:740] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.176427  9893 consensus_queue.cc:260] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.179634  9852 raft_consensus.cc:2468] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:51.182209  9894 tablet_bootstrap.cc:654] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.184576  9893 ts_tablet_manager.cc:1434] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:28:51.185685  9893 tablet_bootstrap.cc:492] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:51.185793  9892 tablet_bootstrap.cc:492] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:51.186434  9899 raft_consensus.cc:493] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.186954  9899 raft_consensus.cc:515] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.189056  9899 leader_election.cc:290] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.190649  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6abcf18a4624879b263a46ce7a2d1b7" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.190701  9851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6abcf18a4624879b263a46ce7a2d1b7" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:51.191764  9851 raft_consensus.cc:2468] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
W20260709 12:28:51.192261  9669 leader_election.cc:343] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.193650  9668 leader_election.cc:304] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:51.195338  9899 raft_consensus.cc:2804] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.195887  9899 raft_consensus.cc:493] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.196138  9893 tablet_bootstrap.cc:654] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.196442  9899 raft_consensus.cc:3060] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.201038  9598 catalog_manager.cc:5697] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.205829  9899 raft_consensus.cc:515] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.207351  9894 tablet_bootstrap.cc:492] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.207908  9894 ts_tablet_manager.cc:1403] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.013s
I20260709 12:28:51.208978  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6abcf18a4624879b263a46ce7a2d1b7" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
W20260709 12:28:51.210453  9669 leader_election.cc:343] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.210933  9894 raft_consensus.cc:359] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.211664  9894 raft_consensus.cc:385] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.211957  9894 raft_consensus.cc:740] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.212714  9894 consensus_queue.cc:260] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.213958  9899 leader_election.cc:290] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.214939  9894 ts_tablet_manager.cc:1434] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:28:51.215159  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6abcf18a4624879b263a46ce7a2d1b7" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:51.215713  9852 raft_consensus.cc:3060] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.215993  9894 tablet_bootstrap.cc:492] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:51.218744  9893 tablet_bootstrap.cc:492] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.218942  9892 tablet_bootstrap.cc:654] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.219419  9893 ts_tablet_manager.cc:1403] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.008s
I20260709 12:28:51.222730  9893 raft_consensus.cc:359] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.223279  9893 raft_consensus.cc:385] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.223526  9893 raft_consensus.cc:740] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.223763  9852 raft_consensus.cc:2468] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:51.224201  9893 consensus_queue.cc:260] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.225209  9668 leader_election.cc:304] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:51.226617  9893 ts_tablet_manager.cc:1434] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:28:51.227069  9894 tablet_bootstrap.cc:654] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.227528  9893 tablet_bootstrap.cc:492] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:51.228755  9899 raft_consensus.cc:2804] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.230156  9899 raft_consensus.cc:697] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:51.231494  9899 consensus_queue.cc:237] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.234537  9893 tablet_bootstrap.cc:654] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.238888  9892 tablet_bootstrap.cc:492] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.239518  9892 ts_tablet_manager.cc:1403] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.004s
I20260709 12:28:51.242851  9893 tablet_bootstrap.cc:492] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.243321  9893 ts_tablet_manager.cc:1403] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:28:51.243551  9892 raft_consensus.cc:359] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.244474  9892 raft_consensus.cc:385] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.244828  9892 raft_consensus.cc:740] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.245981  9893 raft_consensus.cc:359] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.245831  9892 consensus_queue.cc:260] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.246800  9893 raft_consensus.cc:385] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.247164  9893 raft_consensus.cc:740] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.247961  9893 consensus_queue.cc:260] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.248925  9892 ts_tablet_manager.cc:1434] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:28:51.250010  9893 ts_tablet_manager.cc:1434] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:28:51.250355  9892 tablet_bootstrap.cc:492] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:51.251089  9893 tablet_bootstrap.cc:492] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:51.256376  9894 tablet_bootstrap.cc:492] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.256968  9894 ts_tablet_manager.cc:1403] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.009s
I20260709 12:28:51.259969  9894 raft_consensus.cc:359] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.260437  9601 catalog_manager.cc:5697] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.263321  9893 tablet_bootstrap.cc:654] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.264396  9894 raft_consensus.cc:385] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.264766  9894 raft_consensus.cc:740] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.265580  9894 consensus_queue.cc:260] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.266851  9892 tablet_bootstrap.cc:654] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.267905  9894 ts_tablet_manager.cc:1434] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:28:51.269337  9894 tablet_bootstrap.cc:492] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:51.270926  9898 raft_consensus.cc:493] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.271895  9898 raft_consensus.cc:515] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.274868  9893 tablet_bootstrap.cc:492] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.281472  9893 ts_tablet_manager.cc:1403] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.004s
I20260709 12:28:51.283268  9894 tablet_bootstrap.cc:654] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.284461  9893 raft_consensus.cc:359] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.285233  9893 raft_consensus.cc:385] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.285540  9893 raft_consensus.cc:740] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.287952  9892 tablet_bootstrap.cc:492] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.288529  9892 ts_tablet_manager.cc:1403] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.038s	user 0.004s	sys 0.013s
I20260709 12:28:51.289455  9893 consensus_queue.cc:260] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.290858  9894 tablet_bootstrap.cc:492] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.291631  9894 ts_tablet_manager.cc:1403] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.000s
I20260709 12:28:51.291646  9892 raft_consensus.cc:359] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.292768  9892 raft_consensus.cc:385] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.293123  9892 raft_consensus.cc:740] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.296108  9893 ts_tablet_manager.cc:1434] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:28:51.295601  9894 raft_consensus.cc:359] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.296633  9894 raft_consensus.cc:385] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.296994  9894 raft_consensus.cc:740] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.297223  9893 tablet_bootstrap.cc:492] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:51.295718  9898 leader_election.cc:290] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.297935  9894 consensus_queue.cc:260] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.300818  9894 ts_tablet_manager.cc:1434] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:51.301925  9894 tablet_bootstrap.cc:492] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:51.293926  9892 consensus_queue.cc:260] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.304621  9893 tablet_bootstrap.cc:654] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.306694  9892 ts_tablet_manager.cc:1434] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.002s
W20260709 12:28:51.309222  9649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:51.310726  9894 tablet_bootstrap.cc:654] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.315760  9892 tablet_bootstrap.cc:492] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:51.321851  9893 tablet_bootstrap.cc:492] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.322461  9893 ts_tablet_manager.cc:1403] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.011s
I20260709 12:28:51.325008  9892 tablet_bootstrap.cc:654] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.325474  9893 raft_consensus.cc:359] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.326295  9893 raft_consensus.cc:385] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.326619  9893 raft_consensus.cc:740] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.327435  9893 consensus_queue.cc:260] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.330188  9893 ts_tablet_manager.cc:1434] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:28:51.333396  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:51.335613  9893 tablet_bootstrap.cc:492] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
W20260709 12:28:51.336028  9818 leader_election.cc:343] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.342338  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.342968  9778 raft_consensus.cc:2468] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:51.344237  9818 leader_election.cc:304] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:51.344380  9893 tablet_bootstrap.cc:654] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.346329  9918 raft_consensus.cc:2804] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.346903  9918 raft_consensus.cc:493] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.347306  9918 raft_consensus.cc:3060] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.347941  9894 tablet_bootstrap.cc:492] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.348492  9894 ts_tablet_manager.cc:1403] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.000s
I20260709 12:28:51.351995  9894 raft_consensus.cc:359] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.352723  9894 raft_consensus.cc:385] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.353024  9894 raft_consensus.cc:740] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.353742  9894 consensus_queue.cc:260] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.356070  9918 raft_consensus.cc:515] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.357973  9893 tablet_bootstrap.cc:492] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.358741  9894 ts_tablet_manager.cc:1434] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.003s
I20260709 12:28:51.358896  9893 ts_tablet_manager.cc:1403] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.012s
I20260709 12:28:51.359902  9894 tablet_bootstrap.cc:492] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:51.363574  9893 raft_consensus.cc:359] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.364586  9893 raft_consensus.cc:385] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.365031  9893 raft_consensus.cc:740] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.366230  9893 consensus_queue.cc:260] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.367220  9894 tablet_bootstrap.cc:654] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.371589  9893 ts_tablet_manager.cc:1434] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 12:28:51.371636  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:51.372527  9892 tablet_bootstrap.cc:492] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.372869  9893 tablet_bootstrap.cc:492] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:51.373056  9892 ts_tablet_manager.cc:1403] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.058s	user 0.033s	sys 0.018s
W20260709 12:28:51.373365  9818 leader_election.cc:343] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.375789  9918 leader_election.cc:290] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.376246  9892 raft_consensus.cc:359] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.376952  9892 raft_consensus.cc:385] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.377250  9892 raft_consensus.cc:740] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.377292  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:51.377918  9778 raft_consensus.cc:3060] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.378067  9892 consensus_queue.cc:260] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.379741  9893 tablet_bootstrap.cc:654] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.380447  9892 ts_tablet_manager.cc:1434] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:51.381484  9892 tablet_bootstrap.cc:492] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:51.391080  9892 tablet_bootstrap.cc:654] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.391340  9893 tablet_bootstrap.cc:492] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.391340  9778 raft_consensus.cc:2468] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:51.392153  9893 ts_tablet_manager.cc:1403] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:28:51.392972  9818 leader_election.cc:304] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:51.394394  9918 raft_consensus.cc:2804] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.396106  9918 raft_consensus.cc:697] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:51.396869  9894 tablet_bootstrap.cc:492] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.397426  9894 ts_tablet_manager.cc:1403] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.003s
I20260709 12:28:51.397614  9918 consensus_queue.cc:237] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.400554  9893 raft_consensus.cc:359] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.400667  9894 raft_consensus.cc:359] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.401384  9893 raft_consensus.cc:385] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.401461  9894 raft_consensus.cc:385] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.401755  9893 raft_consensus.cc:740] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.401865  9894 raft_consensus.cc:740] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.402062  9892 tablet_bootstrap.cc:492] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.402710  9893 consensus_queue.cc:260] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.402918  9902 raft_consensus.cc:493] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.403167  9894 consensus_queue.cc:260] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.403755  9902 raft_consensus.cc:515] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.405174  9893 ts_tablet_manager.cc:1434] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:28:51.406581  9893 tablet_bootstrap.cc:492] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:51.409713  9902 leader_election.cc:290] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.402894  9892 ts_tablet_manager.cc:1403] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.006s
I20260709 12:28:51.414338  9892 raft_consensus.cc:359] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.414906  9893 tablet_bootstrap.cc:654] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.415221  9892 raft_consensus.cc:385] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.415716  9892 raft_consensus.cc:740] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.416646  9892 consensus_queue.cc:260] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.422015  9598 catalog_manager.cc:5697] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 0 to 1, leader changed from <none> to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 1 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.423974  9899 raft_consensus.cc:493] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.424531  9899 raft_consensus.cc:515] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.426680  9899 leader_election.cc:290] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.428712  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:51.429695  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.430544  9778 raft_consensus.cc:2393] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05b878c1341641c8b469d8fa1a927143 in current term 1: Already voted for candidate cd46df97a8334a35a9f5bbcd1e47460d in this term.
I20260709 12:28:51.432946  9892 ts_tablet_manager.cc:1434] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:28:51.434077  9669 leader_election.cc:304] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143; no voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:28:51.435954  9899 raft_consensus.cc:3060] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.441093  9892 tablet_bootstrap.cc:492] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:51.443737  9899 raft_consensus.cc:2749] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.449957  9918 raft_consensus.cc:493] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.450721  9918 raft_consensus.cc:515] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.448329  9892 tablet_bootstrap.cc:654] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.454533  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55eb762448eb4e04b202d365b4212711" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:51.455395  9703 raft_consensus.cc:2468] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:51.457050  9818 leader_election.cc:304] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:51.458201  9898 raft_consensus.cc:2804] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.458720  9898 raft_consensus.cc:493] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.459118  9898 raft_consensus.cc:3060] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.471127  9918 leader_election.cc:290] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.472395  9898 raft_consensus.cc:515] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.476296  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55eb762448eb4e04b202d365b4212711" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:51.477061  9703 raft_consensus.cc:3060] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.479657  9702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6abcf18a4624879b263a46ce7a2d1b7" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:51.484350  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55eb762448eb4e04b202d365b4212711" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.485205  9778 raft_consensus.cc:2468] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:51.486287  9898 leader_election.cc:290] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.485962  9703 raft_consensus.cc:2468] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:51.485376  9894 ts_tablet_manager.cc:1434] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.088s	user 0.021s	sys 0.009s
I20260709 12:28:51.488215  9818 leader_election.cc:304] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:51.488464  9894 tablet_bootstrap.cc:492] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:51.489099  9898 raft_consensus.cc:2804] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.490332  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55eb762448eb4e04b202d365b4212711" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:51.490768  9898 raft_consensus.cc:697] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:51.491498  9893 tablet_bootstrap.cc:492] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:51.492000  9893 ts_tablet_manager.cc:1403] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.086s	user 0.035s	sys 0.041s
I20260709 12:28:51.491737  9898 consensus_queue.cc:237] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.492727  9892 tablet_bootstrap.cc:492] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.493283  9892 ts_tablet_manager.cc:1403] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.052s	user 0.022s	sys 0.019s
I20260709 12:28:51.494933  9893 raft_consensus.cc:359] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.495842  9893 raft_consensus.cc:385] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.496227  9893 raft_consensus.cc:740] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.496320  9892 raft_consensus.cc:359] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.497174  9892 raft_consensus.cc:385] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.497506  9892 raft_consensus.cc:740] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.497350  9893 consensus_queue.cc:260] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.498613  9892 consensus_queue.cc:260] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.499907  9893 ts_tablet_manager.cc:1434] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:51.501011  9892 ts_tablet_manager.cc:1434] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:28:51.502074  9892 tablet_bootstrap.cc:492] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:51.505187  9894 tablet_bootstrap.cc:654] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.491020  9778 raft_consensus.cc:3060] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.507254  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6abcf18a4624879b263a46ce7a2d1b7" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:51.508082  9852 raft_consensus.cc:2393] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in current term 1: Already voted for candidate 05b878c1341641c8b469d8fa1a927143 in this term.
I20260709 12:28:51.508687  9892 tablet_bootstrap.cc:654] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.512447  9599 catalog_manager.cc:5697] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 0 to 1, leader changed from <none> to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 1 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.515666  9778 raft_consensus.cc:2468] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:51.519472  9743 leader_election.cc:304] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:28:51.524226  9894 tablet_bootstrap.cc:492] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.524439  9892 tablet_bootstrap.cc:492] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.524797  9894 ts_tablet_manager.cc:1403] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.005s
I20260709 12:28:51.524988  9892 ts_tablet_manager.cc:1403] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 12:28:51.526150  9928 raft_consensus.cc:3060] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.527755  9894 raft_consensus.cc:359] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.528538  9894 raft_consensus.cc:385] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.528931  9894 raft_consensus.cc:740] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.528261  9892 raft_consensus.cc:359] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.529619  9892 raft_consensus.cc:385] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.529973  9892 raft_consensus.cc:740] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.529726  9894 consensus_queue.cc:260] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.530769  9892 consensus_queue.cc:260] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.533018  9892 ts_tablet_manager.cc:1434] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:28:51.533964  9892 tablet_bootstrap.cc:492] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:51.534144  9928 raft_consensus.cc:2749] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.535277  9894 ts_tablet_manager.cc:1434] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:28:51.536309  9894 tablet_bootstrap.cc:492] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:51.539569  9892 tablet_bootstrap.cc:654] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.544817  9894 tablet_bootstrap.cc:654] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.544843  9892 tablet_bootstrap.cc:492] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:51.545632  9892 ts_tablet_manager.cc:1403] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 12:28:51.548367  9892 raft_consensus.cc:359] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.549034  9892 raft_consensus.cc:385] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.549353  9892 raft_consensus.cc:740] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.550014  9892 consensus_queue.cc:260] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.550745  9894 tablet_bootstrap.cc:492] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:51.551268  9894 ts_tablet_manager.cc:1403] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:28:51.551767  9892 ts_tablet_manager.cc:1434] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:28:51.553527  9894 raft_consensus.cc:359] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.554261  9894 raft_consensus.cc:385] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.554531  9894 raft_consensus.cc:740] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.555230  9894 consensus_queue.cc:260] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.557204  9894 ts_tablet_manager.cc:1434] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:28:51.569402  9898 raft_consensus.cc:493] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.569939  9898 raft_consensus.cc:515] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.571811  9898 leader_election.cc:290] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.572485  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c12f2b7ab884baeb47afe3912c5e3e6" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:51.572762  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c12f2b7ab884baeb47afe3912c5e3e6" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.573127  9703 raft_consensus.cc:2468] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:51.573415  9778 raft_consensus.cc:2468] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:51.574103  9818 leader_election.cc:304] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:51.574976  9898 raft_consensus.cc:2804] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.575331  9898 raft_consensus.cc:493] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.575564  9898 raft_consensus.cc:3060] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.577155  9899 raft_consensus.cc:493] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.577601  9899 raft_consensus.cc:515] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.579247  9899 leader_election.cc:290] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.580024  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc9216de2034f97862627395f7661e0" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.580716  9778 raft_consensus.cc:2468] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:51.581558  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc9216de2034f97862627395f7661e0" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:51.581588  9669 leader_election.cc:304] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:51.582089  9852 raft_consensus.cc:2468] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:51.582185  9898 raft_consensus.cc:515] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.582564  9899 raft_consensus.cc:2804] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.583024  9899 raft_consensus.cc:493] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.583349  9899 raft_consensus.cc:3060] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.584194  9898 leader_election.cc:290] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.584868  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c12f2b7ab884baeb47afe3912c5e3e6" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:51.585291  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c12f2b7ab884baeb47afe3912c5e3e6" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:51.585465  9703 raft_consensus.cc:3060] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.585860  9778 raft_consensus.cc:3060] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.588837  9899 raft_consensus.cc:515] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.590631  9899 leader_election.cc:290] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.591462  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc9216de2034f97862627395f7661e0" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:51.592037  9777 raft_consensus.cc:3060] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.592795  9778 raft_consensus.cc:2468] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:51.592945  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc9216de2034f97862627395f7661e0" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:51.593353  9703 raft_consensus.cc:2468] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:51.593564  9852 raft_consensus.cc:3060] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.594353  9818 leader_election.cc:304] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:51.595707  9898 raft_consensus.cc:2804] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.596390  9898 raft_consensus.cc:697] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:51.597229  9898 consensus_queue.cc:237] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.600059  9777 raft_consensus.cc:2468] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:51.600811  9852 raft_consensus.cc:2468] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:51.601500  9669 leader_election.cc:304] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:51.602520  9899 raft_consensus.cc:2804] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.603048  9899 raft_consensus.cc:697] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:51.603968  9898 raft_consensus.cc:493] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.604275  9899 consensus_queue.cc:237] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.604444  9898 raft_consensus.cc:515] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.607101  9898 leader_election.cc:290] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.607344  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:51.607046  9598 catalog_manager.cc:5697] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 0 to 1, leader changed from <none> to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 1 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.607878  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.608109  9703 raft_consensus.cc:2468] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:51.608547  9777 raft_consensus.cc:2468] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:51.609594  9818 leader_election.cc:304] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:51.610996  9898 raft_consensus.cc:2804] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.611467  9898 raft_consensus.cc:493] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.611861  9898 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.615612  9601 catalog_manager.cc:5697] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.618908  9898 raft_consensus.cc:515] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.620796  9898 leader_election.cc:290] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.621430  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:51.621821  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:51.622087  9703 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.622458  9777 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.629211  9777 raft_consensus.cc:2468] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:51.629211  9703 raft_consensus.cc:2468] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:51.630452  9818 leader_election.cc:304] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:51.631036  9898 raft_consensus.cc:2804] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.631431  9898 raft_consensus.cc:697] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:51.632217  9898 consensus_queue.cc:237] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.639299  9601 catalog_manager.cc:5697] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 0 to 1, leader changed from <none> to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 1 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.676381  9899 raft_consensus.cc:493] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.676960  9899 raft_consensus.cc:515] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.678791  9899 leader_election.cc:290] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.679713  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5345b9a348a14191aaddaf426ea23439" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.679937  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5345b9a348a14191aaddaf426ea23439" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:51.680415  9777 raft_consensus.cc:2468] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:51.680567  9852 raft_consensus.cc:2468] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:51.681478  9669 leader_election.cc:304] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:51.682173  9899 raft_consensus.cc:2804] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.682585  9899 raft_consensus.cc:493] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.682861  9899 raft_consensus.cc:3060] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.688380  9899 raft_consensus.cc:515] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.689877  9906 consensus_queue.cc:1048] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.690563  9899 leader_election.cc:290] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.691612  9906 consensus_queue.cc:1048] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.691893  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5345b9a348a14191aaddaf426ea23439" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:51.692343  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5345b9a348a14191aaddaf426ea23439" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:51.692546  9852 raft_consensus.cc:3060] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.693025  9777 raft_consensus.cc:3060] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.701990  9852 raft_consensus.cc:2468] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:51.705753  9777 raft_consensus.cc:2468] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:51.706938  9668 leader_election.cc:304] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:51.708457  9899 raft_consensus.cc:2804] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.715232  9899 raft_consensus.cc:697] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:51.716506  9899 consensus_queue.cc:237] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.721839  9898 raft_consensus.cc:493] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.722594  9898 raft_consensus.cc:515] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.726722  9928 raft_consensus.cc:493] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.726828  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:51.727731  9703 raft_consensus.cc:2468] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:51.727592  9906 consensus_queue.cc:1048] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.729224  9818 leader_election.cc:304] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:51.729514  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:51.730101  9918 raft_consensus.cc:2804] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.730551  9918 raft_consensus.cc:493] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.727536  9928 raft_consensus.cc:515] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.730548  9898 leader_election.cc:290] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.734913  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:51.735513  9703 raft_consensus.cc:2468] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 0.
I20260709 12:28:51.737085  9744 leader_election.cc:304] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:51.738255  9902 raft_consensus.cc:2804] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.738610  9902 raft_consensus.cc:493] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.739092  9902 raft_consensus.cc:3060] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.740381  9945 consensus_queue.cc:1048] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.731320  9918 raft_consensus.cc:3060] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.757336  9928 leader_election.cc:290] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.759676  9851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:51.760767  9902 raft_consensus.cc:515] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.761983  9777 raft_consensus.cc:2393] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd46df97a8334a35a9f5bbcd1e47460d in current term 1: Already voted for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in this term.
I20260709 12:28:51.762115  9598 catalog_manager.cc:5697] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.769605  9902 leader_election.cc:290] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:51.770843  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:51.772465  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:51.773133  9703 raft_consensus.cc:3060] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.781190  9918 raft_consensus.cc:515] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.781801  9703 raft_consensus.cc:2468] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 1.
I20260709 12:28:51.783073  9851 raft_consensus.cc:2393] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in current term 1: Already voted for candidate cd46df97a8334a35a9f5bbcd1e47460d in this term.
I20260709 12:28:51.783841  9744 leader_election.cc:304] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:28:51.785141  9955 raft_consensus.cc:2804] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.785630  9955 raft_consensus.cc:697] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:51.786556  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:51.786471  9955 consensus_queue.cc:237] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:51.787333  9703 raft_consensus.cc:2393] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd46df97a8334a35a9f5bbcd1e47460d in current term 1: Already voted for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in this term.
I20260709 12:28:51.791836  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:51.793653  9818 leader_election.cc:304] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd46df97a8334a35a9f5bbcd1e47460d; no voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:51.794559  9898 raft_consensus.cc:2749] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.798848  9918 leader_election.cc:290] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:51.803979  9601 catalog_manager.cc:5697] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.870407  9918 consensus_queue.cc:1048] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.883282  9951 consensus_queue.cc:1048] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.910082  9951 consensus_queue.cc:1048] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.923543  9898 consensus_queue.cc:1048] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.993369  9898 consensus_queue.cc:1048] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.009673  9944 consensus_queue.cc:1048] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.025556  9949 consensus_queue.cc:1048] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:28:52.029938  9951 consensus_queue.cc:1048] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.043174  9951 consensus_queue.cc:1048] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.047611  9851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
mode: GRACEFUL
new_leader_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:41476
I20260709 12:28:52.048452  9851 raft_consensus.cc:606] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Received request to transfer leadership to TS 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:52.060626  9851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c12f2b7ab884baeb47afe3912c5e3e6"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
mode: GRACEFUL
new_leader_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:41476
I20260709 12:28:52.061231  9851 raft_consensus.cc:606] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Received request to transfer leadership to TS 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:52.066906  9898 consensus_queue.cc:1048] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.068321  9851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55eb762448eb4e04b202d365b4212711"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
mode: GRACEFUL
new_leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:41476
I20260709 12:28:52.068889  9851 raft_consensus.cc:606] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Received request to transfer leadership to TS 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:52.080722  9851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
mode: GRACEFUL
new_leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:41476
I20260709 12:28:52.081197  9851 raft_consensus.cc:606] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Received request to transfer leadership to TS 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:52.093324  9703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfc9216de2034f97862627395f7661e0"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
mode: GRACEFUL
new_leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:57638
I20260709 12:28:52.093801  9703 raft_consensus.cc:606] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Received request to transfer leadership to TS 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:52.098860  9703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5345b9a348a14191aaddaf426ea23439"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
mode: GRACEFUL
new_leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:57638
I20260709 12:28:52.099334  9703 raft_consensus.cc:606] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Received request to transfer leadership to TS 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:52.111770  9777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
mode: GRACEFUL
new_leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
 from {username='slave'} at 127.0.0.1:40190
I20260709 12:28:52.112207  9777 raft_consensus.cc:606] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Received request to transfer leadership to TS cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:28:52.164886  9949 consensus_queue.cc:1048] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.175259  9944 consensus_queue.cc:1048] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.182838  9944 raft_consensus.cc:993] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143: : Instructing follower 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:52.183344  9947 raft_consensus.cc:1081] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Signalling peer 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:52.184835  9777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5345b9a348a14191aaddaf426ea23439"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:60750
I20260709 12:28:52.185447  9777 raft_consensus.cc:493] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.185837  9777 raft_consensus.cc:3060] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.191421  9777 raft_consensus.cc:515] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.193303  9777 leader_election.cc:290] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:52.193938  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5345b9a348a14191aaddaf426ea23439" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:52.194402  9851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5345b9a348a14191aaddaf426ea23439" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:52.194651  9703 raft_consensus.cc:3055] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:52.195039  9851 raft_consensus.cc:3060] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.195073  9703 raft_consensus.cc:740] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:52.196252  9703 consensus_queue.cc:260] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.197570  9703 raft_consensus.cc:3060] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.202364  9851 raft_consensus.cc:2468] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:52.203922  9743 leader_election.cc:304] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:52.204727  9955 raft_consensus.cc:2804] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.205236  9955 raft_consensus.cc:697] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:52.205726  9902 consensus_queue.cc:1048] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.206153  9703 raft_consensus.cc:2468] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:52.206064  9955 consensus_queue.cc:237] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.213275  9902 raft_consensus.cc:993] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26: : Instructing follower cd46df97a8334a35a9f5bbcd1e47460d to start an election
I20260709 12:28:52.213790  9928 raft_consensus.cc:1081] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Signalling peer cd46df97a8334a35a9f5bbcd1e47460d to start an election
I20260709 12:28:52.213862  9600 catalog_manager.cc:5697] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 1 to 2, leader changed from 05b878c1341641c8b469d8fa1a927143 (127.29.37.65) to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.217095  9852 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
 from {username='slave'} at 127.0.0.1:54318
I20260709 12:28:52.217654  9852 raft_consensus.cc:493] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.218055  9852 raft_consensus.cc:3060] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.222152  9928 consensus_queue.cc:1048] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.225628  9852 raft_consensus.cc:515] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.231554  9852 leader_election.cc:290] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 2 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:52.232589  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:52.233208  9703 raft_consensus.cc:3060] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.234067  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb936063bf7442549dcb3a3bf3964d5c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:52.234768  9777 raft_consensus.cc:3055] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:52.235153  9777 raft_consensus.cc:740] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:52.235987  9777 consensus_queue.cc:260] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.237278  9777 raft_consensus.cc:3060] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.241395  9703 raft_consensus.cc:2468] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 2.
I20260709 12:28:52.244019  9818 leader_election.cc:304] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:52.244486  9777 raft_consensus.cc:2468] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 2.
I20260709 12:28:52.244804  9951 raft_consensus.cc:2804] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.245385  9951 raft_consensus.cc:697] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 2 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:52.246243  9951 consensus_queue.cc:237] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.252038  9598 catalog_manager.cc:5697] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 1 to 2, leader changed from 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66) to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 2 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.323868  9918 raft_consensus.cc:993] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d: : Instructing follower 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:52.324411  9951 raft_consensus.cc:1081] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Signalling peer 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:52.325863  9777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55eb762448eb4e04b202d365b4212711"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:60756
I20260709 12:28:52.326390  9777 raft_consensus.cc:493] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.326710  9777 raft_consensus.cc:3060] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.331735  9777 raft_consensus.cc:515] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.333202  9777 leader_election.cc:290] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:52.334076  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55eb762448eb4e04b202d365b4212711" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:52.334331  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55eb762448eb4e04b202d365b4212711" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:52.334822  9703 raft_consensus.cc:3060] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.334867  9852 raft_consensus.cc:3055] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:52.335326  9852 raft_consensus.cc:740] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:52.336166  9852 consensus_queue.cc:260] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.337419  9852 raft_consensus.cc:3060] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.342052  9703 raft_consensus.cc:2468] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:52.343448  9744 leader_election.cc:304] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:52.344153  9928 raft_consensus.cc:2804] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.344482  9852 raft_consensus.cc:2468] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:52.344681  9928 raft_consensus.cc:697] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:52.345626  9928 consensus_queue.cc:237] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.351936  9598 catalog_manager.cc:5697] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 1 to 2, leader changed from cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67) to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.418378  9918 raft_consensus.cc:993] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d: : Instructing follower 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:52.418882  9951 raft_consensus.cc:1081] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Signalling peer 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:52.420320  9703 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c12f2b7ab884baeb47afe3912c5e3e6"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:49036
I20260709 12:28:52.420912  9703 raft_consensus.cc:493] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.421306  9703 raft_consensus.cc:3060] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.426345  9703 raft_consensus.cc:515] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.427958  9703 leader_election.cc:290] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:52.428762  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c12f2b7ab884baeb47afe3912c5e3e6" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:52.428998  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c12f2b7ab884baeb47afe3912c5e3e6" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:52.429469  9777 raft_consensus.cc:3060] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.429749  9852 raft_consensus.cc:3055] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:52.430156  9852 raft_consensus.cc:740] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:52.430972  9852 consensus_queue.cc:260] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.432036  9852 raft_consensus.cc:3060] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.434918  9777 raft_consensus.cc:2468] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:52.436031  9669 leader_election.cc:304] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:52.436745  9944 raft_consensus.cc:2804] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.437135  9944 raft_consensus.cc:697] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:52.438103  9852 raft_consensus.cc:2468] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:52.437981  9944 consensus_queue.cc:237] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.444552  9598 catalog_manager.cc:5697] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 1 to 2, leader changed from cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67) to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 2 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.505594  9918 raft_consensus.cc:993] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d: : Instructing follower 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:52.506100  9951 raft_consensus.cc:1081] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Signalling peer 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:52.507529  9703 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:49036
I20260709 12:28:52.508004  9703 raft_consensus.cc:493] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.508349  9703 raft_consensus.cc:3060] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.513310  9703 raft_consensus.cc:515] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.516203  9703 leader_election.cc:290] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:52.516336  9777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:52.517498  9777 raft_consensus.cc:3060] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.517897  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79af30301c2424b9d62b8caac3fed98" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:52.518832  9852 raft_consensus.cc:3055] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:52.519672  9949 raft_consensus.cc:993] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143: : Instructing follower 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:52.519358  9852 raft_consensus.cc:740] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:52.520166  9949 raft_consensus.cc:1081] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Signalling peer 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:52.521826  9776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfc9216de2034f97862627395f7661e0"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:60750
I20260709 12:28:52.521442  9852 consensus_queue.cc:260] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.522456  9776 raft_consensus.cc:493] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.522936  9776 raft_consensus.cc:3060] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.523401  9852 raft_consensus.cc:3060] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.524505  9777 raft_consensus.cc:2468] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:52.525461  9669 leader_election.cc:304] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:52.526137  9949 raft_consensus.cc:2804] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.526630  9949 raft_consensus.cc:697] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:52.527477  9949 consensus_queue.cc:237] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.530361  9852 raft_consensus.cc:2468] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:52.530361  9776 raft_consensus.cc:515] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.532383  9776 leader_election.cc:290] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:52.533406  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc9216de2034f97862627395f7661e0" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:52.533661  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfc9216de2034f97862627395f7661e0" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:52.534035  9703 raft_consensus.cc:3055] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:52.534351  9852 raft_consensus.cc:3060] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.534493  9703 raft_consensus.cc:740] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:52.535315  9703 consensus_queue.cc:260] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.536273  9703 raft_consensus.cc:3060] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.535643  9598 catalog_manager.cc:5697] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 1 to 2, leader changed from cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67) to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 2 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.540304  9852 raft_consensus.cc:2468] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:52.541350  9743 leader_election.cc:304] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:52.542208  9928 raft_consensus.cc:2804] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.542717  9928 raft_consensus.cc:697] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:52.542524  9703 raft_consensus.cc:2468] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:52.543543  9928 consensus_queue.cc:237] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.551923  9918 raft_consensus.cc:993] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d: : Instructing follower 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:52.552582  9951 raft_consensus.cc:1081] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Signalling peer 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:52.551631  9598 catalog_manager.cc:5697] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 1 to 2, leader changed from 05b878c1341641c8b469d8fa1a927143 (127.29.37.65) to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.554141  9776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:60756
I20260709 12:28:52.554678  9776 raft_consensus.cc:493] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:52.555029  9776 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.561153  9776 raft_consensus.cc:515] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.562803  9776 leader_election.cc:290] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:52.563509  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:52.563719  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:52.564097  9703 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.564280  9852 raft_consensus.cc:3055] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:52.564704  9852 raft_consensus.cc:740] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:52.565524  9852 consensus_queue.cc:260] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.566664  9852 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.569914  9703 raft_consensus.cc:2468] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:52.570960  9744 leader_election.cc:304] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:52.571650  9928 raft_consensus.cc:2804] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:52.572127  9928 raft_consensus.cc:697] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:52.572475  9852 raft_consensus.cc:2468] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:52.572979  9928 consensus_queue.cc:237] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:52.579036  9598 catalog_manager.cc:5697] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 1 to 2, leader changed from cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67) to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.714998  9852 raft_consensus.cc:1275] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.716818  9928 consensus_queue.cc:1048] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:52.727066  9703 raft_consensus.cc:1275] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.731681  9928 consensus_queue.cc:1048] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:52.753090  9703 raft_consensus.cc:1275] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer cd46df97a8334a35a9f5bbcd1e47460d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.754346  9951 consensus_queue.cc:1048] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:52.761842  9776 raft_consensus.cc:1275] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Refusing update from remote peer cd46df97a8334a35a9f5bbcd1e47460d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.763736  9898 consensus_queue.cc:1048] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:52.826009  9852 raft_consensus.cc:1275] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.827315  9928 consensus_queue.cc:1048] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:52.835637  9703 raft_consensus.cc:1275] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.837102  9928 consensus_queue.cc:1048] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:52.923879  9703 raft_consensus.cc:1275] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.925302  9928 consensus_queue.cc:1048] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:52.933113  9852 raft_consensus.cc:1275] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:52.934827  9977 consensus_queue.cc:1048] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:53.007498  9776 raft_consensus.cc:1275] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:53.008774  9949 consensus_queue.cc:1048] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:53.018237  9852 raft_consensus.cc:1275] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:53.019783  9949 consensus_queue.cc:1048] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:53.037117  9852 raft_consensus.cc:1275] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:53.038231  9949 consensus_queue.cc:1048] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:53.046610  9776 raft_consensus.cc:1275] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:53.048164  9980 consensus_queue.cc:1048] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:53.108049  9852 raft_consensus.cc:1275] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:53.109223  9977 consensus_queue.cc:1048] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:53.117045  9703 raft_consensus.cc:1275] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:53.118840  9928 consensus_queue.cc:1048] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:55.137457  9598 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57196:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:55.139523  9598 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:55.188182  9693 tablet_service.cc:1511] Processing CreateTablet for tablet 38a7a503a301413ea43e9b69c6b199f7 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:55.189555  9693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a7a503a301413ea43e9b69c6b199f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.190779  9768 tablet_service.cc:1511] Processing CreateTablet for tablet 38a7a503a301413ea43e9b69c6b199f7 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:55.191592  9839 tablet_service.cc:1511] Processing CreateTablet for tablet 38a7a503a301413ea43e9b69c6b199f7 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:55.192107  9768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a7a503a301413ea43e9b69c6b199f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.192886  9839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38a7a503a301413ea43e9b69c6b199f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.193840  9767 tablet_service.cc:1511] Processing CreateTablet for tablet bb11ac409c774f2e8da0c8e9d71891f5 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:55.195120  9767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb11ac409c774f2e8da0c8e9d71891f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.197381  9690 tablet_service.cc:1511] Processing CreateTablet for tablet bb11ac409c774f2e8da0c8e9d71891f5 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:55.198684  9690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb11ac409c774f2e8da0c8e9d71891f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.207653  9838 tablet_service.cc:1511] Processing CreateTablet for tablet bb11ac409c774f2e8da0c8e9d71891f5 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:55.209062  9838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb11ac409c774f2e8da0c8e9d71891f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.210896  9689 tablet_service.cc:1511] Processing CreateTablet for tablet 2200cb77564746429ae41f01f8727e7f (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:55.211387  9766 tablet_service.cc:1511] Processing CreateTablet for tablet 2200cb77564746429ae41f01f8727e7f (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:55.212215  9689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2200cb77564746429ae41f01f8727e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.212729  9766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2200cb77564746429ae41f01f8727e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.225917  9842 tablet_service.cc:1511] Processing CreateTablet for tablet 2200cb77564746429ae41f01f8727e7f (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:55.227356  9842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2200cb77564746429ae41f01f8727e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.230262  9691 tablet_service.cc:1511] Processing CreateTablet for tablet f5e0d31069f844a586b5bf4057c3cf3a (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:55.231588  9691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5e0d31069f844a586b5bf4057c3cf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.240120  9765 tablet_service.cc:1511] Processing CreateTablet for tablet f5e0d31069f844a586b5bf4057c3cf3a (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:55.241958  9765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5e0d31069f844a586b5bf4057c3cf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.257038  9998 tablet_bootstrap.cc:492] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:55.258719  9999 tablet_bootstrap.cc:492] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:55.258991  9839 tablet_service.cc:1511] Processing CreateTablet for tablet f5e0d31069f844a586b5bf4057c3cf3a (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:55.260459  9839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5e0d31069f844a586b5bf4057c3cf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.264621  9693 tablet_service.cc:1511] Processing CreateTablet for tablet 9e0b120174a14b5ebb19e3c854bd46f4 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:55.265952  9693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e0b120174a14b5ebb19e3c854bd46f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.266446  9998 tablet_bootstrap.cc:654] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.277007  9768 tablet_service.cc:1511] Processing CreateTablet for tablet 9e0b120174a14b5ebb19e3c854bd46f4 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:55.278574  9999 tablet_bootstrap.cc:654] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.290632  9768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e0b120174a14b5ebb19e3c854bd46f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.290179  9838 tablet_service.cc:1511] Processing CreateTablet for tablet 9e0b120174a14b5ebb19e3c854bd46f4 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:55.290463  9691 tablet_service.cc:1511] Processing CreateTablet for tablet 887d9f7f674e43439c5053fa05a78c15 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:55.291641  9838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e0b120174a14b5ebb19e3c854bd46f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.292207  9691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887d9f7f674e43439c5053fa05a78c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.305866  9765 tablet_service.cc:1511] Processing CreateTablet for tablet 887d9f7f674e43439c5053fa05a78c15 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:55.306604 10000 tablet_bootstrap.cc:492] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.307358  9765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887d9f7f674e43439c5053fa05a78c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.308465  9998 tablet_bootstrap.cc:492] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:55.308986  9998 ts_tablet_manager.cc:1403] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.052s	user 0.007s	sys 0.016s
I20260709 12:28:55.312201  9998 raft_consensus.cc:359] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.312954  9998 raft_consensus.cc:385] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.313252  9998 raft_consensus.cc:740] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.314442 10000 tablet_bootstrap.cc:654] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.318111  9998 consensus_queue.cc:260] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.325927  9690 tablet_service.cc:1511] Processing CreateTablet for tablet 4af2e8ba9e4d467193035f3c6bff13ff (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:55.327477  9690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af2e8ba9e4d467193035f3c6bff13ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.328183  9999 tablet_bootstrap.cc:492] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:55.328774  9999 ts_tablet_manager.cc:1403] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.006s
I20260709 12:28:55.328922  9839 tablet_service.cc:1511] Processing CreateTablet for tablet 4af2e8ba9e4d467193035f3c6bff13ff (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:55.330435  9767 tablet_service.cc:1511] Processing CreateTablet for tablet 4af2e8ba9e4d467193035f3c6bff13ff (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:55.331128  9839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af2e8ba9e4d467193035f3c6bff13ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.331679  9767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af2e8ba9e4d467193035f3c6bff13ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.335012  9838 tablet_service.cc:1511] Processing CreateTablet for tablet 887d9f7f674e43439c5053fa05a78c15 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:55.337967  9998 ts_tablet_manager.cc:1434] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:28:55.340262  9838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887d9f7f674e43439c5053fa05a78c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.346015  9691 tablet_service.cc:1511] Processing CreateTablet for tablet 91de123bb35f424bb7e4c300518d2b75 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:55.347616  9691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91de123bb35f424bb7e4c300518d2b75. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:55.350026  9649 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:55.360792  9766 tablet_service.cc:1511] Processing CreateTablet for tablet 91de123bb35f424bb7e4c300518d2b75 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:55.362237  9766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91de123bb35f424bb7e4c300518d2b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.362498  9842 tablet_service.cc:1511] Processing CreateTablet for tablet 91de123bb35f424bb7e4c300518d2b75 (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:55.363775  9842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91de123bb35f424bb7e4c300518d2b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.370528  9693 tablet_service.cc:1511] Processing CreateTablet for tablet ef43007c2e8d40758d50bea2cf26ae2c (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:55.371865  9693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef43007c2e8d40758d50bea2cf26ae2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.383045  9998 tablet_bootstrap.cc:492] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:55.384516  9764 tablet_service.cc:1511] Processing CreateTablet for tablet ef43007c2e8d40758d50bea2cf26ae2c (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:55.385874  9764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef43007c2e8d40758d50bea2cf26ae2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.386281  9840 tablet_service.cc:1511] Processing CreateTablet for tablet ef43007c2e8d40758d50bea2cf26ae2c (DEFAULT_TABLE table=soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:55.387578  9840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef43007c2e8d40758d50bea2cf26ae2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:55.402997 10000 tablet_bootstrap.cc:492] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:55.403707 10000 ts_tablet_manager.cc:1403] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.042s
I20260709 12:28:55.407215 10000 raft_consensus.cc:359] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.407588  9918 raft_consensus.cc:493] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.412465  9918 raft_consensus.cc:515] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.414604  9918 leader_election.cc:290] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:55.416055  9702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:55.416327  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:55.415704 10000 raft_consensus.cc:385] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.417111 10000 raft_consensus.cc:740] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
W20260709 12:28:55.417891  9818 leader_election.cc:343] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.417819 10000 consensus_queue.cc:260] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.412199  9999 raft_consensus.cc:359] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.420056  9999 raft_consensus.cc:385] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.420131 10000 ts_tablet_manager.cc:1434] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:28:55.420342  9999 raft_consensus.cc:740] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.421285 10000 tablet_bootstrap.cc:492] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.421077  9999 consensus_queue.cc:260] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.422704  9702 raft_consensus.cc:2468] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:55.424068  9999 ts_tablet_manager.cc:1434] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.057s	user 0.003s	sys 0.006s
I20260709 12:28:55.425196  9999 tablet_bootstrap.cc:492] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:55.427250  9818 leader_election.cc:304] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:55.428443  9918 raft_consensus.cc:2804] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.428797  9918 raft_consensus.cc:493] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.429138  9918 raft_consensus.cc:3060] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.411942  9998 tablet_bootstrap.cc:654] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.435491  9999 tablet_bootstrap.cc:654] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.442364 10000 tablet_bootstrap.cc:654] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.447997  9998 tablet_bootstrap.cc:492] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:55.448582  9998 ts_tablet_manager.cc:1403] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.012s
I20260709 12:28:55.449986  9999 tablet_bootstrap.cc:492] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:55.451154  9999 ts_tablet_manager.cc:1403] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.017s
I20260709 12:28:55.451748  9918 raft_consensus.cc:515] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.451787  9998 raft_consensus.cc:359] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.452764  9998 raft_consensus.cc:385] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.453135  9998 raft_consensus.cc:740] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.454067  9998 consensus_queue.cc:260] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.454298  9999 raft_consensus.cc:359] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.454783  9702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:55.455054  9999 raft_consensus.cc:385] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.455473  9702 raft_consensus.cc:3060] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.456121  9918 leader_election.cc:290] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:55.455431  9999 raft_consensus.cc:740] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.455506 10000 tablet_bootstrap.cc:492] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:55.457688 10000 ts_tablet_manager.cc:1403] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.004s
I20260709 12:28:55.457357  9999 consensus_queue.cc:260] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.459062  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:55.459918  9999 ts_tablet_manager.cc:1434] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:28:55.460911  9999 tablet_bootstrap.cc:492] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:55.460820 10000 raft_consensus.cc:359] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
W20260709 12:28:55.461524  9818 leader_election.cc:343] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.461694 10000 raft_consensus.cc:385] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.462021 10000 raft_consensus.cc:740] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.462908 10000 consensus_queue.cc:260] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.464136 10007 raft_consensus.cc:493] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.464627 10007 raft_consensus.cc:515] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.465224 10000 ts_tablet_manager.cc:1434] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:55.466763  9998 ts_tablet_manager.cc:1434] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:28:55.466784 10007 leader_election.cc:290] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:55.468060  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:55.469070 10000 tablet_bootstrap.cc:492] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.471405  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:55.471776  9999 tablet_bootstrap.cc:654] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:55.475385  9818 leader_election.cc:343] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.475572  9998 tablet_bootstrap.cc:492] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
W20260709 12:28:55.476279  9818 leader_election.cc:343] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.476807  9818 leader_election.cc:304] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd46df97a8334a35a9f5bbcd1e47460d; no voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:55.477619 10007 raft_consensus.cc:2749] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.476297 10000 tablet_bootstrap.cc:654] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.485116  9998 tablet_bootstrap.cc:654] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.501971  9702 raft_consensus.cc:2468] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:55.503808  9818 leader_election.cc:304] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:55.505434 10007 raft_consensus.cc:2804] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.506063 10007 raft_consensus.cc:697] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:55.506705  9998 tablet_bootstrap.cc:492] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:55.507422  9998 ts_tablet_manager.cc:1403] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.007s
I20260709 12:28:55.507217 10007 consensus_queue.cc:237] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.511814  9998 raft_consensus.cc:359] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.512943  9998 raft_consensus.cc:385] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.513406  9998 raft_consensus.cc:740] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.514786  9998 consensus_queue.cc:260] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.518433  9998 ts_tablet_manager.cc:1434] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:28:55.520133  9998 tablet_bootstrap.cc:492] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:55.524042  9598 catalog_manager.cc:5697] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 0 to 1, leader changed from <none> to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 1 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:55.529073  9998 tablet_bootstrap.cc:654] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.535115 10000 tablet_bootstrap.cc:492] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:55.535494  9999 tablet_bootstrap.cc:492] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:55.535701 10000 ts_tablet_manager.cc:1403] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.067s	user 0.019s	sys 0.042s
I20260709 12:28:55.536074  9999 ts_tablet_manager.cc:1403] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.075s	user 0.023s	sys 0.045s
I20260709 12:28:55.539124 10000 raft_consensus.cc:359] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.539980 10000 raft_consensus.cc:385] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.540342 10000 raft_consensus.cc:740] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.540788  9998 tablet_bootstrap.cc:492] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:55.540342  9999 raft_consensus.cc:359] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.541342  9999 raft_consensus.cc:385] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.541361  9998 ts_tablet_manager.cc:1403] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 12:28:55.541785  9999 raft_consensus.cc:740] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.541633 10000 consensus_queue.cc:260] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.542760  9999 consensus_queue.cc:260] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.545187  9999 ts_tablet_manager.cc:1434] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:28:55.546247  9999 tablet_bootstrap.cc:492] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:55.552737  9999 tablet_bootstrap.cc:654] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.552321  9998 raft_consensus.cc:359] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.553301  9998 raft_consensus.cc:385] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.553700  9998 raft_consensus.cc:740] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.554644  9998 consensus_queue.cc:260] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.557262  9998 ts_tablet_manager.cc:1434] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:28:55.558269  9998 tablet_bootstrap.cc:492] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:55.560003 10000 ts_tablet_manager.cc:1434] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.004s
I20260709 12:28:55.561096 10000 tablet_bootstrap.cc:492] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.567392  9949 raft_consensus.cc:493] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.567673 10000 tablet_bootstrap.cc:654] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.567963  9949 raft_consensus.cc:515] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.570886 10007 raft_consensus.cc:493] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.571313  9949 leader_election.cc:290] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.571537 10007 raft_consensus.cc:515] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.573482  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:55.574348  9776 raft_consensus.cc:2468] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:55.575238 10007 leader_election.cc:290] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:55.575585  9702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:55.576282  9702 raft_consensus.cc:2468] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:55.575911  9669 leader_election.cc:304] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:55.575240  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:55.578044  9852 raft_consensus.cc:2468] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:55.580818  9818 leader_election.cc:304] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:55.581228  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:55.581660 10007 raft_consensus.cc:2804] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.581820  9776 raft_consensus.cc:2468] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:55.582072 10007 raft_consensus.cc:493] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.582472 10007 raft_consensus.cc:3060] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.583102 10000 tablet_bootstrap.cc:492] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:55.583624 10000 ts_tablet_manager.cc:1403] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.000s
I20260709 12:28:55.587138 10000 raft_consensus.cc:359] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.587885 10000 raft_consensus.cc:385] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.588208 10000 raft_consensus.cc:740] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.588961 10000 consensus_queue.cc:260] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.591476 10000 ts_tablet_manager.cc:1434] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:55.592106 10007 raft_consensus.cc:515] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.592525 10000 tablet_bootstrap.cc:492] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.592623  9999 tablet_bootstrap.cc:492] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:55.577318 10015 raft_consensus.cc:2804] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.593575 10015 raft_consensus.cc:493] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.593945 10015 raft_consensus.cc:3060] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.593317  9999 ts_tablet_manager.cc:1403] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.047s	user 0.027s	sys 0.006s
I20260709 12:28:55.627632  9928 raft_consensus.cc:493] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.629109  9928 raft_consensus.cc:515] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.634351  9928 leader_election.cc:290] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.637840  9928 raft_consensus.cc:493] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.638975  9928 raft_consensus.cc:515] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.643268  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:55.644639  9703 raft_consensus.cc:2393] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in current term 1: Already voted for candidate cd46df97a8334a35a9f5bbcd1e47460d in this term.
I20260709 12:28:55.644819 10000 tablet_bootstrap.cc:654] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.651520  9702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:55.655953  9998 tablet_bootstrap.cc:654] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.657274  9999 raft_consensus.cc:359] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.658378  9999 raft_consensus.cc:385] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.658810  9999 raft_consensus.cc:740] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.660061  9999 consensus_queue.cc:260] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.660905 10007 leader_election.cc:290] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:55.661423  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
W20260709 12:28:55.662086  9744 leader_election.cc:343] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.664342  9999 ts_tablet_manager.cc:1434] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.004s
I20260709 12:28:55.666496  9999 tablet_bootstrap.cc:492] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:55.666671  9928 leader_election.cc:290] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.671698  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:55.672608  9852 raft_consensus.cc:2468] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 0.
I20260709 12:28:55.675678  9851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:55.677881 10000 tablet_bootstrap.cc:492] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:55.678820 10000 ts_tablet_manager.cc:1403] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.087s	user 0.017s	sys 0.002s
I20260709 12:28:55.683139 10000 raft_consensus.cc:359] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.683617  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:55.688160 10000 raft_consensus.cc:385] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.688474  9776 raft_consensus.cc:3060] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.688755 10000 raft_consensus.cc:740] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.689851  9999 tablet_bootstrap.cc:654] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.686916 10015 raft_consensus.cc:515] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.699437  9703 raft_consensus.cc:2393] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd46df97a8334a35a9f5bbcd1e47460d in current term 1: Already voted for candidate 05b878c1341641c8b469d8fa1a927143 in this term.
I20260709 12:28:55.701797  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:55.702860  9852 raft_consensus.cc:2393] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 05b878c1341641c8b469d8fa1a927143 in current term 1: Already voted for candidate cd46df97a8334a35a9f5bbcd1e47460d in this term.
I20260709 12:28:55.690140 10000 consensus_queue.cc:260] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.686801  9743 leader_election.cc:304] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:55.704813 10017 raft_consensus.cc:2804] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.705014 10000 ts_tablet_manager.cc:1434] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.004s
I20260709 12:28:55.705252 10017 raft_consensus.cc:493] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.703224  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:55.705729 10017 raft_consensus.cc:3060] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.706779 10000 tablet_bootstrap.cc:492] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.707139  9743 leader_election.cc:304] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:28:55.708065  9928 raft_consensus.cc:3060] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.710769  9668 leader_election.cc:304] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143; no voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:28:55.711834 10015 leader_election.cc:290] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.713124  9949 raft_consensus.cc:2749] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.719151  9998 tablet_bootstrap.cc:492] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:55.720088  9998 ts_tablet_manager.cc:1403] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.162s	user 0.019s	sys 0.002s
I20260709 12:28:55.729729  9998 raft_consensus.cc:359] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.722648 10000 tablet_bootstrap.cc:654] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.724460 10017 raft_consensus.cc:515] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.736174 10017 leader_election.cc:290] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.719991  9928 raft_consensus.cc:2749] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.736622  9998 raft_consensus.cc:385] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.737483  9998 raft_consensus.cc:740] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.737838  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:55.738524  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:55.738617  9852 raft_consensus.cc:3060] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.738369  9998 consensus_queue.cc:260] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.739121  9703 raft_consensus.cc:3060] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.747645  9998 ts_tablet_manager.cc:1434] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:28:55.749119  9998 tablet_bootstrap.cc:492] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:55.749305  9703 raft_consensus.cc:2468] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 1.
I20260709 12:28:55.750653  9744 leader_election.cc:304] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:55.751840 10017 raft_consensus.cc:2804] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.752391 10017 raft_consensus.cc:697] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:55.753821 10017 consensus_queue.cc:237] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.764691  9852 raft_consensus.cc:2468] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 1.
I20260709 12:28:55.773375  9998 tablet_bootstrap.cc:654] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.776539  9999 tablet_bootstrap.cc:492] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:55.776809  9776 raft_consensus.cc:2468] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:55.777136  9999 ts_tablet_manager.cc:1403] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.111s	user 0.028s	sys 0.028s
I20260709 12:28:55.778712  9818 leader_election.cc:304] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:55.779743 10007 raft_consensus.cc:2804] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.780400 10007 raft_consensus.cc:697] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:55.781780 10007 consensus_queue.cc:237] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.782867  9999 raft_consensus.cc:359] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.783744  9999 raft_consensus.cc:385] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.784070  9999 raft_consensus.cc:740] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.784853  9999 consensus_queue.cc:260] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.791754  9601 catalog_manager.cc:5697] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 0 to 1, leader changed from <none> to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 1 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:55.791776  9999 ts_tablet_manager.cc:1434] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 12:28:55.797640  9999 tablet_bootstrap.cc:492] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:55.798722 10000 tablet_bootstrap.cc:492] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:55.799301 10000 ts_tablet_manager.cc:1403] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.017s
I20260709 12:28:55.802950 10000 raft_consensus.cc:359] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.803778 10000 raft_consensus.cc:385] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.804075 10000 raft_consensus.cc:740] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.804774  9998 tablet_bootstrap.cc:492] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:55.805789  9999 tablet_bootstrap.cc:654] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.807852  9998 ts_tablet_manager.cc:1403] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.007s
I20260709 12:28:55.804968 10000 consensus_queue.cc:260] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.811430 10000 ts_tablet_manager.cc:1434] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 12:28:55.813542 10000 tablet_bootstrap.cc:492] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.818720  9998 raft_consensus.cc:359] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.819459  9998 raft_consensus.cc:385] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.819788  9998 raft_consensus.cc:740] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.820609  9998 consensus_queue.cc:260] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.822207 10000 tablet_bootstrap.cc:654] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.822764  9998 ts_tablet_manager.cc:1434] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.005s
I20260709 12:28:55.824661 10007 raft_consensus.cc:493] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.825273 10007 raft_consensus.cc:515] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.827662 10007 leader_election.cc:290] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:55.828337  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:55.828773  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:55.826306  9598 catalog_manager.cc:5697] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:55.830096  9818 leader_election.cc:343] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:55.831043  9818 leader_election.cc:343] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.831568  9818 leader_election.cc:304] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd46df97a8334a35a9f5bbcd1e47460d; no voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:55.832880 10007 raft_consensus.cc:2749] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.833212  9998 tablet_bootstrap.cc:492] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:55.839037 10019 raft_consensus.cc:493] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.836192  9999 tablet_bootstrap.cc:492] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:55.839488 10000 tablet_bootstrap.cc:492] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:55.839955  9999 ts_tablet_manager.cc:1403] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.014s
I20260709 12:28:55.843708  9999 raft_consensus.cc:359] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.844424  9999 raft_consensus.cc:385] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.844713  9999 raft_consensus.cc:740] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.845362  9998 tablet_bootstrap.cc:654] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.839778 10019 raft_consensus.cc:515] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.840065 10000 ts_tablet_manager.cc:1403] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.006s
I20260709 12:28:55.850128  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5e0d31069f844a586b5bf4057c3cf3a" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:55.850071 10000 raft_consensus.cc:359] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.850932  9776 raft_consensus.cc:2468] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:55.851183 10000 raft_consensus.cc:385] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.851650 10000 raft_consensus.cc:740] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.852468  9669 leader_election.cc:304] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:55.853351  9949 raft_consensus.cc:2804] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.852681 10000 consensus_queue.cc:260] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.853722  9949 raft_consensus.cc:493] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.846017  9999 consensus_queue.cc:260] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.854117  9949 raft_consensus.cc:3060] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.857055  9999 ts_tablet_manager.cc:1434] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:28:55.858665  9999 tablet_bootstrap.cc:492] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:55.859057  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5e0d31069f844a586b5bf4057c3cf3a" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:55.862498  9852 raft_consensus.cc:2468] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:55.859335 10023 raft_consensus.cc:493] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.863351 10023 raft_consensus.cc:515] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.865617 10023 leader_election.cc:290] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.866734  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5e0d31069f844a586b5bf4057c3cf3a" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:55.859581 10019 leader_election.cc:290] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.861728 10000 ts_tablet_manager.cc:1434] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.005s
I20260709 12:28:55.868027  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5e0d31069f844a586b5bf4057c3cf3a" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:55.868641 10000 tablet_bootstrap.cc:492] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.868650  9852 raft_consensus.cc:2468] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 0.
I20260709 12:28:55.877054  9743 leader_election.cc:304] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:55.878727 10023 raft_consensus.cc:2804] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.878966 10000 tablet_bootstrap.cc:654] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.879093 10023 raft_consensus.cc:493] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.879642 10023 raft_consensus.cc:3060] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.883971  9999 tablet_bootstrap.cc:654] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.888813 10023 raft_consensus.cc:515] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.892417  9702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5e0d31069f844a586b5bf4057c3cf3a" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:55.892235  9949 raft_consensus.cc:515] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.893469  9703 raft_consensus.cc:2393] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in current term 1: Already voted for candidate 05b878c1341641c8b469d8fa1a927143 in this term.
I20260709 12:28:55.894927  9928 raft_consensus.cc:493] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.895500  9928 raft_consensus.cc:515] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.895955  9949 leader_election.cc:290] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.897233  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5e0d31069f844a586b5bf4057c3cf3a" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:55.897827  9852 raft_consensus.cc:3060] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.899824  9998 tablet_bootstrap.cc:492] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:55.900537  9998 ts_tablet_manager.cc:1403] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.076s	user 0.018s	sys 0.042s
I20260709 12:28:55.903070  9949 raft_consensus.cc:493] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.903566  9949 raft_consensus.cc:515] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.903919  9928 leader_election.cc:290] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.905666 10023 leader_election.cc:290] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.904554  9998 raft_consensus.cc:359] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.906637  9998 raft_consensus.cc:385] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.906956  9998 raft_consensus.cc:740] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.907045  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5e0d31069f844a586b5bf4057c3cf3a" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:55.907682 10000 tablet_bootstrap.cc:492] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:55.907955  9776 raft_consensus.cc:2393] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 05b878c1341641c8b469d8fa1a927143 in current term 1: Already voted for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in this term.
I20260709 12:28:55.908288 10000 ts_tablet_manager.cc:1403] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.002s
I20260709 12:28:55.907786  9998 consensus_queue.cc:260] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.912089  9851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e0b120174a14b5ebb19e3c854bd46f4" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:55.912742  9851 raft_consensus.cc:2468] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 0.
I20260709 12:28:55.913916  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5e0d31069f844a586b5bf4057c3cf3a" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:55.915896  9743 leader_election.cc:304] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:55.916317  9949 leader_election.cc:290] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.916965  9743 leader_election.cc:304] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:28:55.917824 10024 raft_consensus.cc:2804] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.918522 10007 consensus_queue.cc:1048] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:55.919270 10024 raft_consensus.cc:493] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.919652 10024 raft_consensus.cc:3060] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.921244  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e0b120174a14b5ebb19e3c854bd46f4" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:55.921854  9703 raft_consensus.cc:2468] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 0.
I20260709 12:28:55.922643 10023 raft_consensus.cc:2749] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.918037  9852 raft_consensus.cc:2468] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:55.923535  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:55.924134  9998 ts_tablet_manager.cc:1434] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
I20260709 12:28:55.924151  9850 raft_consensus.cc:2468] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:55.924605  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:55.924742  9668 leader_election.cc:304] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:55.925393  9778 raft_consensus.cc:2468] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:55.925568  9998 tablet_bootstrap.cc:492] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:55.926106  9668 leader_election.cc:304] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:55.926502 10019 raft_consensus.cc:2804] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.926970 10015 raft_consensus.cc:2804] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.927023 10019 raft_consensus.cc:697] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:55.927448 10015 raft_consensus.cc:493] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.927899 10015 raft_consensus.cc:3060] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.928233 10019 consensus_queue.cc:237] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.933459  9998 tablet_bootstrap.cc:654] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.911419 10000 raft_consensus.cc:359] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.935768 10000 raft_consensus.cc:385] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.936056 10000 raft_consensus.cc:740] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.936797  9999 tablet_bootstrap.cc:492] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:55.937361  9999 ts_tablet_manager.cc:1403] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.079s	user 0.025s	sys 0.001s
I20260709 12:28:55.940665  9999 raft_consensus.cc:359] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.941394  9999 raft_consensus.cc:385] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.941928  9999 raft_consensus.cc:740] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.942801  9999 consensus_queue.cc:260] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.945286  9999 ts_tablet_manager.cc:1434] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:28:55.945031 10015 raft_consensus.cc:515] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.946609  9999 tablet_bootstrap.cc:492] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:55.947410 10000 consensus_queue.cc:260] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.953905 10000 ts_tablet_manager.cc:1434] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 12:28:55.955506 10000 tablet_bootstrap.cc:492] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26: Bootstrap starting.
I20260709 12:28:55.961064 10015 leader_election.cc:290] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.962450  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:55.963308  9776 raft_consensus.cc:3060] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.966883 10024 raft_consensus.cc:515] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:55.967411  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:55.968242  9850 raft_consensus.cc:3060] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.972640 10024 leader_election.cc:290] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:55.982353  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e0b120174a14b5ebb19e3c854bd46f4" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:55.983219  9852 raft_consensus.cc:3060] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.986698  9701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e0b120174a14b5ebb19e3c854bd46f4" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:55.987365  9701 raft_consensus.cc:3060] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.991393  9776 raft_consensus.cc:2468] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:55.991894 10007 consensus_queue.cc:1048] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:55.992684  9669 leader_election.cc:304] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:55.993629 10015 raft_consensus.cc:2804] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.994294 10015 raft_consensus.cc:697] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:55.995167 10015 consensus_queue.cc:237] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.005767 10000 tablet_bootstrap.cc:654] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.010829  9850 raft_consensus.cc:2468] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:56.018290  9852 raft_consensus.cc:2468] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 1.
I20260709 12:28:56.020932  9999 tablet_bootstrap.cc:654] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.020545  9599 catalog_manager.cc:5697] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.029613  9743 leader_election.cc:304] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:56.031010  9701 raft_consensus.cc:2468] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 1.
I20260709 12:28:56.031155 10024 raft_consensus.cc:2804] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:56.032935 10024 raft_consensus.cc:697] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:56.037499  9998 tablet_bootstrap.cc:492] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:56.038038  9998 ts_tablet_manager.cc:1403] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.113s	user 0.020s	sys 0.027s
I20260709 12:28:56.040318 10024 consensus_queue.cc:237] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.041533  9998 raft_consensus.cc:359] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.042244  9998 raft_consensus.cc:385] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.042518  9998 raft_consensus.cc:740] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.055553  9998 consensus_queue.cc:260] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.056977  9599 catalog_manager.cc:5697] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.058058  9998 ts_tablet_manager.cc:1434] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:28:56.059373  9998 tablet_bootstrap.cc:492] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d: Bootstrap starting.
I20260709 12:28:56.064164  9999 tablet_bootstrap.cc:492] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:56.064726  9999 ts_tablet_manager.cc:1403] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.008s
I20260709 12:28:56.067492  9999 raft_consensus.cc:359] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.068313  9999 raft_consensus.cc:385] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.068634  9999 raft_consensus.cc:740] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.069515  9999 consensus_queue.cc:260] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.071805  9999 ts_tablet_manager.cc:1434] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:28:56.072888  9999 tablet_bootstrap.cc:492] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143: Bootstrap starting.
I20260709 12:28:56.074048  9998 tablet_bootstrap.cc:654] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.079645  9999 tablet_bootstrap.cc:654] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.081014 10019 raft_consensus.cc:493] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.081689 10019 raft_consensus.cc:515] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.085357  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af2e8ba9e4d467193035f3c6bff13ff" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:56.089202  9776 raft_consensus.cc:2468] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:56.090600  9669 leader_election.cc:304] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:56.091746 10015 raft_consensus.cc:2804] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:56.092228 10015 raft_consensus.cc:493] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.092648 10015 raft_consensus.cc:3060] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.095288 10019 leader_election.cc:290] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.097316  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af2e8ba9e4d467193035f3c6bff13ff" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:56.098066  9852 raft_consensus.cc:2468] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
W20260709 12:28:56.118327 10040 log.cc:927] Time spent T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:28:56.131321 10000 tablet_bootstrap.cc:492] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26: No bootstrap required, opened a new log
I20260709 12:28:56.131489  9998 tablet_bootstrap.cc:492] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d: No bootstrap required, opened a new log
I20260709 12:28:56.131898 10000 ts_tablet_manager.cc:1403] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent bootstrapping tablet: real 0.177s	user 0.014s	sys 0.009s
I20260709 12:28:56.132418  9998 ts_tablet_manager.cc:1403] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d: Time spent bootstrapping tablet: real 0.073s	user 0.019s	sys 0.015s
I20260709 12:28:56.134712 10000 raft_consensus.cc:359] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.135869 10000 raft_consensus.cc:385] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.136180 10000 raft_consensus.cc:740] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.136008  9998 raft_consensus.cc:359] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.136785  9998 raft_consensus.cc:385] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.137064  9998 raft_consensus.cc:740] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.136996 10000 consensus_queue.cc:260] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.137887  9998 consensus_queue.cc:260] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.138949 10000 ts_tablet_manager.cc:1434] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:56.139315  9599 catalog_manager.cc:5697] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 1 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.141386 10015 raft_consensus.cc:515] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.142606  9998 ts_tablet_manager.cc:1434] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:28:56.144291  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af2e8ba9e4d467193035f3c6bff13ff" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:56.144450 10015 leader_election.cc:290] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.144992  9778 raft_consensus.cc:3060] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.148885  9999 tablet_bootstrap.cc:492] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143: No bootstrap required, opened a new log
I20260709 12:28:56.149417  9999 ts_tablet_manager.cc:1403] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143: Time spent bootstrapping tablet: real 0.077s	user 0.017s	sys 0.042s
I20260709 12:28:56.152204  9999 raft_consensus.cc:359] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.152494  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af2e8ba9e4d467193035f3c6bff13ff" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:56.152851  9999 raft_consensus.cc:385] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.153107 10007 raft_consensus.cc:493] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.153646 10007 raft_consensus.cc:515] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.155349  9778 raft_consensus.cc:2468] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:56.155764 10007 leader_election.cc:290] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:56.153101  9852 raft_consensus.cc:3060] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.156502  9669 leader_election.cc:304] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:56.156468  9701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43007c2e8d40758d50bea2cf26ae2c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143" is_pre_election: true
I20260709 12:28:56.157498 10015 raft_consensus.cc:2804] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:56.158272 10015 raft_consensus.cc:697] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:56.158527  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43007c2e8d40758d50bea2cf26ae2c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:56.159222 10015 consensus_queue.cc:237] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.159641  9778 raft_consensus.cc:2468] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:56.163055  9852 raft_consensus.cc:2468] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:56.163967  9818 leader_election.cc:304] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:56.153168  9999 raft_consensus.cc:740] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.165064 10007 raft_consensus.cc:2804] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:56.165465 10007 raft_consensus.cc:493] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.165270  9999 consensus_queue.cc:260] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.165843 10007 raft_consensus.cc:3060] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.166298  9701 raft_consensus.cc:2468] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 0.
I20260709 12:28:56.169386  9999 ts_tablet_manager.cc:1434] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:28:56.170107  9598 catalog_manager.cc:5697] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.170708 10015 raft_consensus.cc:493] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.171298 10015 raft_consensus.cc:515] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.174178 10015 leader_election.cc:290] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.174059  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91de123bb35f424bb7e4c300518d2b75" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" is_pre_election: true
I20260709 12:28:56.174530  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91de123bb35f424bb7e4c300518d2b75" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" is_pre_election: true
I20260709 12:28:56.175091  9778 raft_consensus.cc:2468] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:56.175580  9852 raft_consensus.cc:2468] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 0.
I20260709 12:28:56.176390  9669 leader_election.cc:304] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:56.173959 10007 raft_consensus.cc:515] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.177610 10019 raft_consensus.cc:2804] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:56.177974 10019 raft_consensus.cc:493] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.178323 10019 raft_consensus.cc:3060] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.178891  9701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43007c2e8d40758d50bea2cf26ae2c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:56.179224 10007 leader_election.cc:290] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:28:56.179509  9701 raft_consensus.cc:3060] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.183018  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef43007c2e8d40758d50bea2cf26ae2c" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:56.183617  9778 raft_consensus.cc:3060] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.186976  9701 raft_consensus.cc:2468] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:56.188166  9818 leader_election.cc:304] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:56.189041 10007 raft_consensus.cc:2804] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:56.189536 10007 raft_consensus.cc:697] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:56.190368 10007 consensus_queue.cc:237] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.191000  9778 raft_consensus.cc:2468] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 1.
I20260709 12:28:56.192864 10019 raft_consensus.cc:515] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.195127 10019 leader_election.cc:290] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.196098  9852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91de123bb35f424bb7e4c300518d2b75" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:56.196576  9852 raft_consensus.cc:3060] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.196225  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91de123bb35f424bb7e4c300518d2b75" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:56.197137  9778 raft_consensus.cc:3060] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.201588  9852 raft_consensus.cc:2468] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:56.202634  9668 leader_election.cc:304] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:56.203392 10019 raft_consensus.cc:2804] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:56.203860 10019 raft_consensus.cc:697] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:56.204670  9778 raft_consensus.cc:2468] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 1.
I20260709 12:28:56.204665 10019 consensus_queue.cc:237] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.207032  9918 consensus_queue.cc:1048] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:56.211040  9598 catalog_manager.cc:5697] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 0 to 1, leader changed from <none> to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 1 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:56.214357  9599 catalog_manager.cc:5697] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 1 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.224959 10051 consensus_queue.cc:1048] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:56.254408 10024 consensus_queue.cc:1048] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.267768 10023 consensus_queue.cc:1048] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.347723 10019 consensus_queue.cc:1048] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.359797  9949 consensus_queue.cc:1048] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.420950  9852 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
mode: GRACEFUL
new_leader_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:41476
I20260709 12:28:56.421484  9852 raft_consensus.cc:606] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Received request to transfer leadership to TS 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:56.426311  9778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
mode: GRACEFUL
new_leader_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:40190
I20260709 12:28:56.426839  9778 raft_consensus.cc:606] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Received request to transfer leadership to TS 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:56.432994  9852 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
mode: GRACEFUL
new_leader_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:41476
I20260709 12:28:56.433508  9852 raft_consensus.cc:606] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Received request to transfer leadership to TS 05b878c1341641c8b469d8fa1a927143
I20260709 12:28:56.443818  9701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
mode: GRACEFUL
new_leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:57638
I20260709 12:28:56.444326  9701 raft_consensus.cc:606] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Received request to transfer leadership to TS 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:56.448897  9701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4af2e8ba9e4d467193035f3c6bff13ff"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
mode: GRACEFUL
new_leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:57638
I20260709 12:28:56.449368  9701 raft_consensus.cc:606] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Received request to transfer leadership to TS 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:56.453722  9701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91de123bb35f424bb7e4c300518d2b75"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
mode: GRACEFUL
new_leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:57638
I20260709 12:28:56.454149  9701 raft_consensus.cc:606] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Received request to transfer leadership to TS 55e8e5b806c24ad1ad5e4bdf55326c26
I20260709 12:28:56.481817 10023 consensus_queue.cc:1048] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.488803 10019 consensus_queue.cc:1048] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.493283  9928 consensus_queue.cc:1048] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.497000 10019 raft_consensus.cc:993] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: : Instructing follower 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:56.497448 10019 raft_consensus.cc:1081] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Signalling peer 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:56.502509  9776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:60750
I20260709 12:28:56.503129  9776 raft_consensus.cc:493] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:56.503494  9776 raft_consensus.cc:3060] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.509411 10019 consensus_queue.cc:1048] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.516520  9776 raft_consensus.cc:515] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.519533  9701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:56.520690  9701 raft_consensus.cc:3055] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.521086  9701 raft_consensus.cc:740] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:56.520146  9778 raft_consensus.cc:1240] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Rejecting Update request from peer 05b878c1341641c8b469d8fa1a927143 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:56.521998  9776 leader_election.cc:290] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.522785  9701 consensus_queue.cc:260] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.523619  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887d9f7f674e43439c5053fa05a78c15" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:56.524324  9850 raft_consensus.cc:3060] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.524428  9701 raft_consensus.cc:3060] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.530162  9701 raft_consensus.cc:2468] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:56.531454  9744 leader_election.cc:304] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:56.531863  9850 raft_consensus.cc:2468] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:56.532243  9928 raft_consensus.cc:2804] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:56.532799  9928 raft_consensus.cc:697] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:56.533653  9928 consensus_queue.cc:237] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.541815  9600 catalog_manager.cc:5697] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 1 to 2, leader changed from 05b878c1341641c8b469d8fa1a927143 (127.29.37.65) to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.550599 10051 raft_consensus.cc:993] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: : Instructing follower 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:56.551062 10051 raft_consensus.cc:1081] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Signalling peer 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:56.552532  9701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:49036
I20260709 12:28:56.553102  9701 raft_consensus.cc:493] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:56.553468  9701 raft_consensus.cc:3060] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.558307  9701 raft_consensus.cc:515] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.559876  9701 leader_election.cc:290] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.560611  9778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:56.561030  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38a7a503a301413ea43e9b69c6b199f7" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:56.561317  9778 raft_consensus.cc:3060] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.561679  9850 raft_consensus.cc:3055] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.562028  9850 raft_consensus.cc:740] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:56.562870  9850 consensus_queue.cc:260] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.564172  9850 raft_consensus.cc:3060] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.568815  9778 raft_consensus.cc:2468] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:56.569146 10019 consensus_queue.cc:1048] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.570240  9669 leader_election.cc:304] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:56.571055 10019 raft_consensus.cc:2804] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:56.571527  9850 raft_consensus.cc:2468] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:56.571548 10019 raft_consensus.cc:697] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:56.572794 10019 consensus_queue.cc:237] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.577589  9949 raft_consensus.cc:993] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143: : Instructing follower 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:56.578042  9949 raft_consensus.cc:1081] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Signalling peer 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:56.580621  9776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4af2e8ba9e4d467193035f3c6bff13ff"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:60750
I20260709 12:28:56.581489  9776 raft_consensus.cc:493] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:56.581926  9776 raft_consensus.cc:3060] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.582876  9599 catalog_manager.cc:5697] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 1 to 2, leader changed from cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67) to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 2 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.583698 10019 consensus_queue.cc:1048] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.586942  9918 consensus_queue.cc:1048] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.591353  9776 raft_consensus.cc:515] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.594406  9702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af2e8ba9e4d467193035f3c6bff13ff" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:56.594722  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af2e8ba9e4d467193035f3c6bff13ff" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:56.594977  9702 raft_consensus.cc:3055] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.595356  9702 raft_consensus.cc:740] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:56.595407  9850 raft_consensus.cc:3060] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.596233  9702 consensus_queue.cc:260] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.597324  9702 raft_consensus.cc:3060] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.603595  9850 raft_consensus.cc:2468] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:56.604887  9743 leader_election.cc:304] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:56.605762  9928 raft_consensus.cc:2804] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:56.606273  9928 raft_consensus.cc:697] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:56.607097  9928 consensus_queue.cc:237] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.610502  9776 leader_election.cc:290] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.614586  9702 raft_consensus.cc:2468] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:56.614961  9599 catalog_manager.cc:5697] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 1 to 2, leader changed from 05b878c1341641c8b469d8fa1a927143 (127.29.37.65) to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.621728 10007 consensus_queue.cc:1048] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.645263  9928 raft_consensus.cc:993] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: : Instructing follower 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:56.645721 10023 raft_consensus.cc:1081] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Signalling peer 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:56.647104  9702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:49048
I20260709 12:28:56.647650  9702 raft_consensus.cc:493] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:56.648020  9702 raft_consensus.cc:3060] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.655014  9702 raft_consensus.cc:515] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.656679  9702 leader_election.cc:290] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.657318  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:56.657716  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb11ac409c774f2e8da0c8e9d71891f5" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:56.658025  9776 raft_consensus.cc:3055] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.658383  9850 raft_consensus.cc:3060] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.658409  9776 raft_consensus.cc:740] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:56.659255  9776 consensus_queue.cc:260] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.660197  9776 raft_consensus.cc:3060] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.663347  9850 raft_consensus.cc:2468] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:56.664319  9668 leader_election.cc:304] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:56.664958  9776 raft_consensus.cc:2468] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:56.665031 10019 raft_consensus.cc:2804] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:56.665596 10019 raft_consensus.cc:697] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:56.666394 10019 consensus_queue.cc:237] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.673270  9600 catalog_manager.cc:5697] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 1 to 2, leader changed from 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66) to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 2 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.714999 10051 raft_consensus.cc:993] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d: : Instructing follower 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:56.715423 10007 raft_consensus.cc:1081] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Signalling peer 05b878c1341641c8b469d8fa1a927143 to start an election
I20260709 12:28:56.716792  9702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
 from {username='slave'} at 127.0.0.1:49036
I20260709 12:28:56.717262  9702 raft_consensus.cc:493] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:56.717557  9702 raft_consensus.cc:3060] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.724520  9702 raft_consensus.cc:515] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.726696  9702 leader_election.cc:290] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.727700  9776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:28:56.728358  9776 raft_consensus.cc:3060] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.728438  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2200cb77564746429ae41f01f8727e7f" candidate_uuid: "05b878c1341641c8b469d8fa1a927143" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:56.729418  9850 raft_consensus.cc:3055] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.729777  9850 raft_consensus.cc:740] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:28:56.731189  9850 consensus_queue.cc:260] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.732790  9850 raft_consensus.cc:3060] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.734077  9776 raft_consensus.cc:2468] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:56.735298  9669 leader_election.cc:304] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, 55e8e5b806c24ad1ad5e4bdf55326c26; no voters: 
I20260709 12:28:56.736162 10019 raft_consensus.cc:2804] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:56.736642 10019 raft_consensus.cc:697] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Becoming Leader. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:56.737485 10019 consensus_queue.cc:237] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.740800  9850 raft_consensus.cc:2468] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b878c1341641c8b469d8fa1a927143 in term 2.
I20260709 12:28:56.745487  9600 catalog_manager.cc:5697] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 reported cstate change: term changed from 1 to 2, leader changed from cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67) to 05b878c1341641c8b469d8fa1a927143 (127.29.37.65). New cstate: current_term: 2 leader_uuid: "05b878c1341641c8b469d8fa1a927143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.781302 10019 consensus_queue.cc:1048] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.793195 10062 consensus_queue.cc:1048] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.801088 10019 raft_consensus.cc:993] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143: : Instructing follower 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:56.801611 10062 raft_consensus.cc:1081] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Signalling peer 55e8e5b806c24ad1ad5e4bdf55326c26 to start an election
I20260709 12:28:56.803066  9776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91de123bb35f424bb7e4c300518d2b75"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
 from {username='slave'} at 127.0.0.1:60750
I20260709 12:28:56.803591  9776 raft_consensus.cc:493] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:56.803915  9776 raft_consensus.cc:3060] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.808998  9776 raft_consensus.cc:515] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.810431  9776 leader_election.cc:290] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67:40277)
I20260709 12:28:56.811293  9702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91de123bb35f424bb7e4c300518d2b75" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:28:56.811501  9850 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91de123bb35f424bb7e4c300518d2b75" candidate_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
I20260709 12:28:56.811978  9702 raft_consensus.cc:3055] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.812140  9850 raft_consensus.cc:3060] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.812377  9702 raft_consensus.cc:740] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:28:56.813277  9702 consensus_queue.cc:260] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.814599  9702 raft_consensus.cc:3060] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.818632  9850 raft_consensus.cc:2468] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:56.819645  9743 leader_election.cc:304] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:28:56.820495  9928 raft_consensus.cc:2804] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:56.821098  9928 raft_consensus.cc:697] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Becoming Leader. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:28:56.821658  9702 raft_consensus.cc:2468] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55e8e5b806c24ad1ad5e4bdf55326c26 in term 2.
I20260709 12:28:56.821870  9928 consensus_queue.cc:237] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:28:56.828044  9599 catalog_manager.cc:5697] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 reported cstate change: term changed from 1 to 2, leader changed from 05b878c1341641c8b469d8fa1a927143 (127.29.37.65) to 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66). New cstate: current_term: 2 leader_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.993968  9850 raft_consensus.cc:1275] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:56.995293 10023 consensus_queue.cc:1048] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.005642  9701 raft_consensus.cc:1275] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.007186 10023 consensus_queue.cc:1048] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.064637  9701 raft_consensus.cc:1275] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.066036  9928 consensus_queue.cc:1048] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.075258  9850 raft_consensus.cc:1275] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.076781 10023 consensus_queue.cc:1048] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.084379  9852 raft_consensus.cc:1275] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.085767 10062 consensus_queue.cc:1048] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.094357  9776 raft_consensus.cc:1275] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.095942 10015 consensus_queue.cc:1048] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.207376  9776 raft_consensus.cc:1275] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.208683 10015 consensus_queue.cc:1048] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.216357  9852 raft_consensus.cc:1275] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.217720 10062 consensus_queue.cc:1048] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.236100  9776 raft_consensus.cc:1275] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.237192 10019 consensus_queue.cc:1048] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.250684  9852 raft_consensus.cc:1275] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 05b878c1341641c8b469d8fa1a927143: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.252859 10015 consensus_queue.cc:1048] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.286995  9701 raft_consensus.cc:1275] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.288266 10017 consensus_queue.cc:1048] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.296651  9852 raft_consensus.cc:1275] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Refusing update from remote peer 55e8e5b806c24ad1ad5e4bdf55326c26: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:57.298151 10070 consensus_queue.cc:1048] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.459527  9599 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57196:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:28:59.478574  9599 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57196:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:28:59.499365 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:59.719503 10111 log.cc:927] Time spent T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:28:59.756565 10124 tablet.cc:1724] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:59.783025 10120 log.cc:927] Time spent T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:28:59.812862 10131 tablet.cc:1724] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.805308 10130 tablet.cc:1724] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.832984 10141 tablet.cc:1724] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.835237 10137 tablet.cc:1724] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:59.850471 10128 log.cc:927] Time spent T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:28:59.871290 10138 tablet.cc:1724] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.879134  9703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "343c84975f3847e1a1dfd9af9a5ad009"
dest_uuid: "05b878c1341641c8b469d8fa1a927143"
mode: GRACEFUL
new_leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
 from {username='slave'} at 127.0.0.1:57646
I20260709 12:28:59.879998  9703 raft_consensus.cc:606] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Received request to transfer leadership to TS cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:28:59.876966 10136 tablet.cc:1724] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.790529 10127 tablet.cc:1724] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.892361 10144 tablet.cc:1724] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.848078 10133 tablet.cc:1724] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.932391 10143 tablet.cc:1724] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.932662 10124 tablet.cc:1724] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.933317 10135 tablet.cc:1724] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.919679 10137 tablet.cc:1724] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.947481 10124 tablet.cc:1724] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.968355 10124 tablet.cc:1724] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.979522 10141 tablet.cc:1724] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.981742 10137 tablet.cc:1724] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.979974 10143 tablet.cc:1724] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:00.003906 10144 tablet.cc:1724] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:00.008157 10133 tablet.cc:1724] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:59.989456 10131 tablet.cc:1724] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:00.022940 10130 tablet.cc:1724] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:00.023474 10141 tablet.cc:1724] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:00.028458 10143 tablet.cc:1724] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:00.006909 10138 tablet.cc:1724] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:00.045989  9775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601"
dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
mode: GRACEFUL
new_leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
 from {username='slave'} at 127.0.0.1:40200
I20260709 12:29:00.046641  9775 raft_consensus.cc:606] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Received request to transfer leadership to TS cd46df97a8334a35a9f5bbcd1e47460d
I20260709 12:29:00.049391 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:00.050025 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:00.053583 10136 tablet.cc:1724] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:00.073522  9600 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517: soft_deleted_table [id=4981c68f3a5a430fbe31f8ba6cd353e1] alter complete (version 1)
I20260709 12:29:00.132167 10104 raft_consensus.cc:993] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143: : Instructing follower cd46df97a8334a35a9f5bbcd1e47460d to start an election
I20260709 12:29:00.132774 10147 raft_consensus.cc:1081] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Signalling peer cd46df97a8334a35a9f5bbcd1e47460d to start an election
I20260709 12:29:00.134506  9845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "343c84975f3847e1a1dfd9af9a5ad009"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
 from {username='slave'} at 127.0.0.1:54302
I20260709 12:29:00.134977  9845 raft_consensus.cc:493] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:00.135286  9845 raft_consensus.cc:3060] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.141059  9845 raft_consensus.cc:515] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:29:00.142911  9845 leader_election.cc:290] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 2 election: Requested vote from peers 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899), 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227)
I20260709 12:29:00.143569  9775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343c84975f3847e1a1dfd9af9a5ad009" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:29:00.144078  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "343c84975f3847e1a1dfd9af9a5ad009" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:29:00.144302  9775 raft_consensus.cc:3060] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.144716  9703 raft_consensus.cc:3055] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:00.145072  9703 raft_consensus.cc:740] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05b878c1341641c8b469d8fa1a927143, State: Running, Role: LEADER
I20260709 12:29:00.145848  9703 consensus_queue.cc:260] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:29:00.146865  9703 raft_consensus.cc:3060] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.149857  9775 raft_consensus.cc:2468] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 2.
I20260709 12:29:00.151039  9818 leader_election.cc:304] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e8e5b806c24ad1ad5e4bdf55326c26, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:29:00.151501  9703 raft_consensus.cc:2468] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 2.
I20260709 12:29:00.152130 10102 raft_consensus.cc:2804] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:00.152891 10102 raft_consensus.cc:697] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:29:00.153743 10102 consensus_queue.cc:237] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:29:00.162024  9599 catalog_manager.cc:5697] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 1 to 2, leader changed from 05b878c1341641c8b469d8fa1a927143 (127.29.37.65) to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 2 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:00.430806 10152 raft_consensus.cc:993] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26: : Instructing follower cd46df97a8334a35a9f5bbcd1e47460d to start an election
I20260709 12:29:00.431471 10100 raft_consensus.cc:1081] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Signalling peer cd46df97a8334a35a9f5bbcd1e47460d to start an election
I20260709 12:29:00.432842  9845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601"
dest_uuid: "cd46df97a8334a35a9f5bbcd1e47460d"
 from {username='slave'} at 127.0.0.1:54318
I20260709 12:29:00.433327  9845 raft_consensus.cc:493] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:00.433634  9845 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:00.438714  9845 raft_consensus.cc:515] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:29:00.440466  9845 leader_election.cc:290] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 3 election: Requested vote from peers 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227), 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899)
I20260709 12:29:00.441115  9703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05b878c1341641c8b469d8fa1a927143"
I20260709 12:29:00.441852  9703 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:00.442607  9775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93627256cc2d4c94bfabcc2508dce601" candidate_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26"
I20260709 12:29:00.443236  9775 raft_consensus.cc:3055] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:00.443549  9775 raft_consensus.cc:740] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 55e8e5b806c24ad1ad5e4bdf55326c26, State: Running, Role: LEADER
I20260709 12:29:00.444370  9775 consensus_queue.cc:260] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:29:00.445995  9775 raft_consensus.cc:3060] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:00.449594  9703 raft_consensus.cc:2468] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 3.
I20260709 12:29:00.450902  9818 leader_election.cc:304] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b878c1341641c8b469d8fa1a927143, cd46df97a8334a35a9f5bbcd1e47460d; no voters: 
I20260709 12:29:00.451725 10102 raft_consensus.cc:2804] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:00.453140  9775 raft_consensus.cc:2468] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd46df97a8334a35a9f5bbcd1e47460d in term 3.
I20260709 12:29:00.454437 10102 raft_consensus.cc:697] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 3 LEADER]: Becoming Leader. State: Replica: cd46df97a8334a35a9f5bbcd1e47460d, State: Running, Role: LEADER
I20260709 12:29:00.455303 10102 consensus_queue.cc:237] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } }
I20260709 12:29:00.462172  9599 catalog_manager.cc:5697] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d reported cstate change: term changed from 2 to 3, leader changed from 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66) to cd46df97a8334a35a9f5bbcd1e47460d (127.29.37.67). New cstate: current_term: 3 leader_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd46df97a8334a35a9f5bbcd1e47460d" member_type: VOTER last_known_addr { host: "127.29.37.67" port: 40277 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:00.609629  9703 raft_consensus.cc:1275] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Refusing update from remote peer cd46df97a8334a35a9f5bbcd1e47460d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:00.610860 10102 consensus_queue.cc:1048] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:00.619990  9775 raft_consensus.cc:1275] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Refusing update from remote peer cd46df97a8334a35a9f5bbcd1e47460d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:00.621459 10153 consensus_queue.cc:1048] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:00.890553  9703 raft_consensus.cc:1275] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 3 FOLLOWER]: Refusing update from remote peer cd46df97a8334a35a9f5bbcd1e47460d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:00.891701 10153 consensus_queue.cc:1048] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b878c1341641c8b469d8fa1a927143" member_type: VOTER last_known_addr { host: "127.29.37.65" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:00.901459  9775 raft_consensus.cc:1275] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 3 FOLLOWER]: Refusing update from remote peer cd46df97a8334a35a9f5bbcd1e47460d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:00.903331 10102 consensus_queue.cc:1048] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e8e5b806c24ad1ad5e4bdf55326c26" member_type: VOTER last_known_addr { host: "127.29.37.66" port: 32899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:01.051038 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:01.056447 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:01.056823 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:02.057772 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:02.063213 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:02.063632 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:03.064615 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:03.070849 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:03.071453 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:04.072355 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:04.077314 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:04.077739 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:05.078815 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:05.083715 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:05.084136 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:06.085322 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:06.092098 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:06.092556 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:07.093706 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:07.100802 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:07.101307 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:08.102464 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:08.108721 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:08.109122 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:09.110020 29845 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:09.115808 29845 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:09.116168 29845 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:10.173743 29845 tablet_server.cc:179] TabletServer@127.29.37.65:0 shutting down...
W20260709 12:29:10.198283  9744 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 96 similar messages]
W20260709 12:29:10.202011  9744 consensus_peers.cc:597] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 -> Peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Couldn't send request to peer 05b878c1341641c8b469d8fa1a927143. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.218492 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:10.219521 29845 tablet_replica.cc:333] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.221083 29845 raft_consensus.cc:2243] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.222060 29845 raft_consensus.cc:2272] T 91de123bb35f424bb7e4c300518d2b75 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.225556 29845 tablet_replica.cc:333] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.226500 29845 raft_consensus.cc:2243] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:10.227064  9744 consensus_peers.cc:597] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 -> Peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Couldn't send request to peer 05b878c1341641c8b469d8fa1a927143. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.227142 29845 raft_consensus.cc:2272] T 4af2e8ba9e4d467193035f3c6bff13ff P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.230082 29845 tablet_replica.cc:333] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.230753 29845 raft_consensus.cc:2243] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.231352 29845 raft_consensus.cc:2272] T 9e0b120174a14b5ebb19e3c854bd46f4 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.233996 29845 tablet_replica.cc:333] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.234733 29845 raft_consensus.cc:2243] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.235651 29845 raft_consensus.cc:2272] T f5e0d31069f844a586b5bf4057c3cf3a P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.238459 29845 tablet_replica.cc:333] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.239162 29845 raft_consensus.cc:2243] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.240161 29845 raft_consensus.cc:2272] T 2200cb77564746429ae41f01f8727e7f P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.242688 29845 tablet_replica.cc:333] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.243311 29845 raft_consensus.cc:2243] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.244415 29845 raft_consensus.cc:2272] T bb11ac409c774f2e8da0c8e9d71891f5 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.247637 29845 tablet_replica.cc:333] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.248248 29845 raft_consensus.cc:2243] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.249157 29845 raft_consensus.cc:2272] T 38a7a503a301413ea43e9b69c6b199f7 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.252369 29845 tablet_replica.cc:333] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.253119 29845 raft_consensus.cc:2243] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.253860 29845 raft_consensus.cc:2272] T 93627256cc2d4c94bfabcc2508dce601 P 05b878c1341641c8b469d8fa1a927143 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.257666 29845 tablet_replica.cc:333] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.258277 29845 raft_consensus.cc:2243] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.258826 29845 raft_consensus.cc:2272] T 343c84975f3847e1a1dfd9af9a5ad009 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.261286 29845 tablet_replica.cc:333] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.261886 29845 raft_consensus.cc:2243] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.262444 29845 raft_consensus.cc:2272] T 5345b9a348a14191aaddaf426ea23439 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.264631 29845 tablet_replica.cc:333] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.265210 29845 raft_consensus.cc:2243] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.265758 29845 raft_consensus.cc:2272] T 55eb762448eb4e04b202d365b4212711 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.268118 29845 tablet_replica.cc:333] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.268712 29845 raft_consensus.cc:2243] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.269719 29845 raft_consensus.cc:2272] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.272272 29845 tablet_replica.cc:333] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.272871 29845 raft_consensus.cc:2243] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.273375 29845 raft_consensus.cc:2272] T fb936063bf7442549dcb3a3bf3964d5c P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.275813 29845 tablet_replica.cc:333] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.276391 29845 raft_consensus.cc:2243] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.276973 29845 raft_consensus.cc:2272] T ef43007c2e8d40758d50bea2cf26ae2c P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.279488 29845 tablet_replica.cc:333] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.280040 29845 raft_consensus.cc:2243] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.280596 29845 raft_consensus.cc:2272] T 887d9f7f674e43439c5053fa05a78c15 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.282984 29845 tablet_replica.cc:333] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.283562 29845 raft_consensus.cc:2243] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.284035 29845 raft_consensus.cc:2272] T dfc9216de2034f97862627395f7661e0 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.286475 29845 tablet_replica.cc:333] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.286993 29845 raft_consensus.cc:2243] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.287853 29845 raft_consensus.cc:2272] T d79af30301c2424b9d62b8caac3fed98 P 05b878c1341641c8b469d8fa1a927143 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.290059 29845 tablet_replica.cc:333] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143: stopping tablet replica
I20260709 12:29:10.290616 29845 raft_consensus.cc:2243] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.291502 29845 raft_consensus.cc:2272] T e6abcf18a4624879b263a46ce7a2d1b7 P 05b878c1341641c8b469d8fa1a927143 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:10.333251  9744 consensus_peers.cc:597] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 -> Peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Couldn't send request to peer 05b878c1341641c8b469d8fa1a927143. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:41227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:10.337405  9744 consensus_peers.cc:597] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 -> Peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Couldn't send request to peer 05b878c1341641c8b469d8fa1a927143. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:41227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.338423 29845 tablet_server.cc:196] TabletServer@127.29.37.65:0 shutdown complete.
W20260709 12:29:10.364114  9744 consensus_peers.cc:597] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 -> Peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Couldn't send request to peer 05b878c1341641c8b469d8fa1a927143. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:41227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.386505 29845 tablet_server.cc:179] TabletServer@127.29.37.66:0 shutting down...
I20260709 12:29:10.419595 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:10.420533 29845 tablet_replica.cc:333] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.421639 29845 raft_consensus.cc:2243] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.422995 29845 raft_consensus.cc:2272] T 91de123bb35f424bb7e4c300518d2b75 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:10.425199  9818 consensus_peers.cc:597] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d -> Peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Couldn't send request to peer 05b878c1341641c8b469d8fa1a927143. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:41227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.426170 29845 tablet_replica.cc:333] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.426956 29845 raft_consensus.cc:2243] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.427770 29845 raft_consensus.cc:2272] T 4af2e8ba9e4d467193035f3c6bff13ff P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.430176 29845 tablet_replica.cc:333] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.430877 29845 raft_consensus.cc:2243] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.431733 29845 raft_consensus.cc:2272] T 9e0b120174a14b5ebb19e3c854bd46f4 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.434043 29845 tablet_replica.cc:333] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.434719 29845 raft_consensus.cc:2243] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.435278 29845 raft_consensus.cc:2272] T f5e0d31069f844a586b5bf4057c3cf3a P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.437496 29845 tablet_replica.cc:333] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.438081 29845 raft_consensus.cc:2243] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.438761 29845 raft_consensus.cc:2272] T bb11ac409c774f2e8da0c8e9d71891f5 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.440984 29845 tablet_replica.cc:333] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.441632 29845 raft_consensus.cc:2243] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.442092 29845 raft_consensus.cc:2272] T 38a7a503a301413ea43e9b69c6b199f7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.444279 29845 tablet_replica.cc:333] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.444847 29845 raft_consensus.cc:2243] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.445313 29845 raft_consensus.cc:2272] T 2200cb77564746429ae41f01f8727e7f P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.447567 29845 tablet_replica.cc:333] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.448057 29845 raft_consensus.cc:2243] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.448441 29845 raft_consensus.cc:2272] T 93627256cc2d4c94bfabcc2508dce601 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.450655 29845 tablet_replica.cc:333] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.451233 29845 raft_consensus.cc:2243] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.451727 29845 raft_consensus.cc:2272] T 343c84975f3847e1a1dfd9af9a5ad009 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.454455 29845 tablet_replica.cc:333] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.455242 29845 raft_consensus.cc:2243] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.456387 29845 raft_consensus.cc:2272] T 5345b9a348a14191aaddaf426ea23439 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.458760 29845 tablet_replica.cc:333] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.459306 29845 raft_consensus.cc:2243] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.459807 29845 raft_consensus.cc:2272] T 5c12f2b7ab884baeb47afe3912c5e3e6 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.462003 29845 tablet_replica.cc:333] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.462574 29845 raft_consensus.cc:2243] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.463454 29845 raft_consensus.cc:2272] T 55eb762448eb4e04b202d365b4212711 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.465680 29845 tablet_replica.cc:333] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.466233 29845 raft_consensus.cc:2243] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.466711 29845 raft_consensus.cc:2272] T fb936063bf7442549dcb3a3bf3964d5c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.468756 29845 tablet_replica.cc:333] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.469267 29845 raft_consensus.cc:2243] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.469913 29845 raft_consensus.cc:2272] T ef43007c2e8d40758d50bea2cf26ae2c P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.472317 29845 tablet_replica.cc:333] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.472832 29845 raft_consensus.cc:2243] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.473701 29845 raft_consensus.cc:2272] T 887d9f7f674e43439c5053fa05a78c15 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.476117 29845 tablet_replica.cc:333] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.476626 29845 raft_consensus.cc:2243] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.477476 29845 raft_consensus.cc:2272] T dfc9216de2034f97862627395f7661e0 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.479681 29845 tablet_replica.cc:333] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.480183 29845 raft_consensus.cc:2243] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.480665 29845 raft_consensus.cc:2272] T e6abcf18a4624879b263a46ce7a2d1b7 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.482867 29845 tablet_replica.cc:333] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26: stopping tablet replica
I20260709 12:29:10.483347 29845 raft_consensus.cc:2243] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.483814 29845 raft_consensus.cc:2272] T d79af30301c2424b9d62b8caac3fed98 P 55e8e5b806c24ad1ad5e4bdf55326c26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.526712 29845 tablet_server.cc:196] TabletServer@127.29.37.66:0 shutdown complete.
W20260709 12:29:10.556071  9818 consensus_peers.cc:597] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d -> Peer 55e8e5b806c24ad1ad5e4bdf55326c26 (127.29.37.66:32899): Couldn't send request to peer 55e8e5b806c24ad1ad5e4bdf55326c26. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.66:32899: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.568775 29845 tablet_server.cc:179] TabletServer@127.29.37.67:0 shutting down...
I20260709 12:29:10.603274 29845 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:10.604053 29845 tablet_replica.cc:333] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.604846 29845 raft_consensus.cc:2243] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.605402 29845 raft_consensus.cc:2272] T 4af2e8ba9e4d467193035f3c6bff13ff P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.607967 29845 tablet_replica.cc:333] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.608701 29845 raft_consensus.cc:2243] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.609180 29845 raft_consensus.cc:2272] T 91de123bb35f424bb7e4c300518d2b75 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.611413 29845 tablet_replica.cc:333] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.611974 29845 raft_consensus.cc:2243] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.612453 29845 raft_consensus.cc:2272] T 9e0b120174a14b5ebb19e3c854bd46f4 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.615466 29845 tablet_replica.cc:333] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.616441 29845 raft_consensus.cc:2243] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.617089 29845 raft_consensus.cc:2272] T f5e0d31069f844a586b5bf4057c3cf3a P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:10.618742  9818 consensus_peers.cc:597] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d -> Peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Couldn't send request to peer 05b878c1341641c8b469d8fa1a927143. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:41227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:10.619352  9818 consensus_peers.cc:597] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d -> Peer 05b878c1341641c8b469d8fa1a927143 (127.29.37.65:41227): Couldn't send request to peer 05b878c1341641c8b469d8fa1a927143. Status: Network error: Client connection negotiation failed: client connection to 127.29.37.65:41227: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.619824 29845 tablet_replica.cc:333] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.620476 29845 raft_consensus.cc:2243] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.621006 29845 raft_consensus.cc:2272] T 2200cb77564746429ae41f01f8727e7f P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.623217 29845 tablet_replica.cc:333] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.623849 29845 raft_consensus.cc:2243] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.624325 29845 raft_consensus.cc:2272] T bb11ac409c774f2e8da0c8e9d71891f5 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.626682 29845 tablet_replica.cc:333] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.627249 29845 raft_consensus.cc:2243] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.627803 29845 raft_consensus.cc:2272] T 38a7a503a301413ea43e9b69c6b199f7 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.630272 29845 tablet_replica.cc:333] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.630913 29845 raft_consensus.cc:2243] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.632071 29845 raft_consensus.cc:2272] T 343c84975f3847e1a1dfd9af9a5ad009 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.634351 29845 tablet_replica.cc:333] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.634862 29845 raft_consensus.cc:2243] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.635377 29845 raft_consensus.cc:2272] T 5345b9a348a14191aaddaf426ea23439 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.637789 29845 tablet_replica.cc:333] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.638415 29845 raft_consensus.cc:2243] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.638917 29845 raft_consensus.cc:2272] T 55eb762448eb4e04b202d365b4212711 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.641127 29845 tablet_replica.cc:333] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.641635 29845 raft_consensus.cc:2243] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.642138 29845 raft_consensus.cc:2272] T 5c12f2b7ab884baeb47afe3912c5e3e6 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.644471 29845 tablet_replica.cc:333] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.644977 29845 raft_consensus.cc:2243] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.645833 29845 raft_consensus.cc:2272] T fb936063bf7442549dcb3a3bf3964d5c P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.647998 29845 tablet_replica.cc:333] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.648537 29845 raft_consensus.cc:2243] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.649392 29845 raft_consensus.cc:2272] T 93627256cc2d4c94bfabcc2508dce601 P cd46df97a8334a35a9f5bbcd1e47460d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.651548 29845 tablet_replica.cc:333] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.652086 29845 raft_consensus.cc:2243] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.652918 29845 raft_consensus.cc:2272] T ef43007c2e8d40758d50bea2cf26ae2c P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.655001 29845 tablet_replica.cc:333] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.655510 29845 raft_consensus.cc:2243] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.656000 29845 raft_consensus.cc:2272] T 887d9f7f674e43439c5053fa05a78c15 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.657927 29845 tablet_replica.cc:333] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.658481 29845 raft_consensus.cc:2243] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.658955 29845 raft_consensus.cc:2272] T dfc9216de2034f97862627395f7661e0 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.660925 29845 tablet_replica.cc:333] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.661393 29845 raft_consensus.cc:2243] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.661865 29845 raft_consensus.cc:2272] T d79af30301c2424b9d62b8caac3fed98 P cd46df97a8334a35a9f5bbcd1e47460d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.663991 29845 tablet_replica.cc:333] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d: stopping tablet replica
I20260709 12:29:10.664453 29845 raft_consensus.cc:2243] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.664875 29845 raft_consensus.cc:2272] T e6abcf18a4624879b263a46ce7a2d1b7 P cd46df97a8334a35a9f5bbcd1e47460d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.705346 29845 tablet_server.cc:196] TabletServer@127.29.37.67:0 shutdown complete.
I20260709 12:29:10.744379 29845 master.cc:562] Master@127.29.37.126:45671 shutting down...
I20260709 12:29:10.766968 29845 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:10.767648 29845 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.767998 29845 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5f37ea631d584360855eabe75c4ff517: stopping tablet replica
I20260709 12:29:10.789287 29845 master.cc:584] Master@127.29.37.126:45671 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (20662 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (44749 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (416589 ms total)
[  PASSED  ] 8 tests.
I20260709 12:29:10.845335 29845 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2350 messages since previous log ~34 seconds ago
I20260709 12:29:10.845598 29845 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2356 messages since previous log ~35 seconds ago
I20260709 12:29:10.845774 29845 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2356 messages since previous log ~35 seconds ago
I20260709 12:29:10.846004 29845 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:29:10.846194 29845 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:29:10.846431 29845 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 10 messages since previous log ~193 seconds ago
I20260709 12:29:10.846748 29845 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 8 messages since previous log ~0 seconds ago
I20260709 12:29:10.847049 29845 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~19 seconds ago
